2024-03-16T09:41:44.906 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T09:41:44.906 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352 -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/orig.config.yaml 2024-03-16T09:41:44.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T09:41:45.064 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352 branch: wip-batrick-testing-20240314.183243 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7602352' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGax3KfyepI1aeFbFR+eZp0QEsHi3koD1PNm2ErUMi7rvCFiNHez0CQndEf2A36EKowle6bvdTOmp9sZynNTLr8= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNS5RuJNA59ZTTXEREeHQDai/Ymcd64jQ6o72DmdFPGMmssVzPols1GzfpZL5Uv+VgPIBD/QmuP+bQEzaHd5LP8= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3p0FOGBulTj9OeasAvuO55RuNz3ACwJ1Ht4MX9AxJ5OyYQhEEjkH3SJUwBMDc+ejCTO72NgDdspvgyYucxxl4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T09:41:45.064 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T09:41:45.065 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T09:41:45.065 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T09:41:45.066 INFO:teuthology.task.internal:Checking packages... 2024-03-16T09:41:45.088 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T09:41:45.088 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T09:41:45.088 INFO:teuthology.packaging:ref: None 2024-03-16T09:41:45.088 INFO:teuthology.packaging:tag: None 2024-03-16T09:41:45.088 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T09:41:45.088 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:41:45.089 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T09:41:45.226 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-16T09:41:45.227 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T09:41:45.235 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T09:41:45.235 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T09:41:45.255 INFO:teuthology.task.internal:Saving configuration 2024-03-16T09:41:45.277 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T09:41:45.283 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:41:45.310 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.062763', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGax3KfyepI1aeFbFR+eZp0QEsHi3koD1PNm2ErUMi7rvCFiNHez0CQndEf2A36EKowle6bvdTOmp9sZynNTLr8='} 2024-03-16T09:41:45.331 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.063897', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNS5RuJNA59ZTTXEREeHQDai/Ymcd64jQ6o72DmdFPGMmssVzPols1GzfpZL5Uv+VgPIBD/QmuP+bQEzaHd5LP8='} 2024-03-16T09:41:45.355 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.061366', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3p0FOGBulTj9OeasAvuO55RuNz3ACwJ1Ht4MX9AxJ5OyYQhEEjkH3SJUwBMDc+ejCTO72NgDdspvgyYucxxl4='} 2024-03-16T09:41:45.355 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T09:41:45.360 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-16T09:41:45.361 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-16T09:41:45.361 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-16T09:41:45.361 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T09:41:45.464 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0b37894160>, signals=[15]) 2024-03-16T09:41:45.465 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T09:41:45.471 INFO:teuthology.task.internal:Opening connections... 2024-03-16T09:41:45.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T09:41:45.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:41:45.582 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-16T09:41:45.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:41:45.668 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-03-16T09:41:45.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:41:45.753 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T09:41:45.760 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-03-16T09:41:45.778 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-03-16T09:41:45.778 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-03-16T09:41:45.834 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-03-16T09:41:45.834 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-03-16T09:41:45.834 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T09:41:45.835 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T09:41:45.836 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-03-16T09:41:45.862 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-16T09:41:45.879 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-16T09:41:45.880 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-16T09:41:45.936 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T09:41:45.937 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-16T09:41:45.937 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T09:41:45.937 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T09:41:45.937 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T09:41:45.937 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-16T09:41:45.962 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-03-16T09:41:45.979 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-03-16T09:41:45.980 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-03-16T09:41:46.037 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T09:41:46.038 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-03-16T09:41:46.038 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T09:41:46.038 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T09:41:46.038 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T09:41:46.038 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-03-16T09:41:46.062 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T09:41:46.072 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T09:41:46.226 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T09:41:46.226 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-03-16T09:41:46.228 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-16T09:41:46.230 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-03-16T09:41:46.247 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T09:41:46.253 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T09:41:46.253 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-03-16T09:41:46.285 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-16T09:41:46.287 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-03-16T09:41:46.380 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T09:41:46.455 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T09:41:46.470 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-16T09:41:46.470 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-16T09:41:46.470 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-16T09:41:46.470 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T09:41:46.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:41:46.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-16T09:41:46.698 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T09:41:46.699 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3845b7ac715a3ff198582638d199a90e30e254da"... 2024-03-16T09:41:46.699 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-03-16T09:41:46.716 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-361.el9.x86_64 2024-03-16T09:41:46.716 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T09:41:46.717 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-16T09:41:46.717 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum upgrade -y linux-firmware 2024-03-16T09:41:48.894 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-03-16T09:41:53.509 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.0 MB 00:02 2024-03-16T09:42:16.966 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 864 kB/s | 19 MB 00:22 2024-03-16T09:42:21.366 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.1 MB 00:00 2024-03-16T09:42:22.801 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-03-16T09:42:23.869 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-03-16T09:42:29.036 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-16T09:42:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-03-16T09:42:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-03-16T09:42:31.481 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 1 Package 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 375 M 2024-03-16T09:42:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-03-16T09:42:40.369 INFO:teuthology.orchestra.run.smithi039.stdout:linux-firmware-20240219-143.el9.noarch.rpm 42 MB/s | 375 MB 00:08 2024-03-16T09:42:40.370 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:42:40.371 INFO:teuthology.orchestra.run.smithi039.stdout:Total 42 MB/s | 375 MB 00:08 2024-03-16T09:42:40.371 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-03-16T09:42:40.458 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-03-16T09:42:40.458 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-03-16T09:42:42.979 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-03-16T09:42:42.980 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-03-16T09:42:49.366 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-03-16T09:42:49.682 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T09:42:49.779 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T09:42:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T09:42:51.020 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:42:51.379 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-03-16T09:42:51.672 INFO:teuthology.task.kernel:Downloading kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-16T09:42:51.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T09:42:51.823 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ 2024-03-16T09:42:51.823 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-16T09:42:51.826 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi039.front.sepia.ceph.com to complete... 2024-03-16T09:43:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16 09:43:03 URL:https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm [336075563/336075563] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-16T09:43:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:FINISHED --2024-03-16 09:43:03-- 2024-03-16T09:43:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:Total wall clock time: 11s 2024-03-16T09:43:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:Downloaded: 1 files, 321M in 11s (28.9 MB/s) 2024-03-16T09:43:03.248 INFO:teuthology.task.kernel:Installing kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-16T09:43:03.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-16T09:43:05.285 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying... ######################################## 2024-03-16T09:43:05.355 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing... ######################################## 2024-03-16T09:43:05.439 INFO:teuthology.orchestra.run.smithi039.stdout:Updating / installing... 2024-03-16T09:43:13.257 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-6.8.0_rc7_g3845b7ac715a-1 ######################################## 2024-03-16T09:43:29.353 INFO:teuthology.orchestra.run.smithi039.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T09:43:52.494 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-16T09:43:52.495 DEBUG:teuthology.orchestra.run.smithi039:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-16T09:43:52.570 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:52.570 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/System.map 2024-03-16T09:43:52.570 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/config 2024-03-16T09:43:52.570 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel 2024-03-16T09:43:52.570 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-16T09:43:52.571 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-16T09:43:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ansi_cprng.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/anubis.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/arc4.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_memcpy.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_pq.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_tx.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_xor.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/raid6test.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authenc.ko 2024-03-16T09:43:52.573 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authencesn.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blake2b_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_common.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/camellia_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast5_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast6_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast_common.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ccm.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cmac.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crc32_generic.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_engine.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_user.ko 2024-03-16T09:43:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cts.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/des_generic.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/dh_generic.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/echainiv.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/essiv.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/fcrypt.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/khazad.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4hc.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/md4.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/michael_mic.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcbc.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcrypt.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/rmd160.ko 2024-03-16T09:43:52.575 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/seed.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/serpent_generic.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tcrypt.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tea.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_common.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_generic.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/vmac.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/wp512.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xcbc.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xor.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xxhash_generic.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_pad.ko 2024-03-16T09:43:52.576 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/custom_method.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/ec_sys.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/platform_profile.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbs.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbshc.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/video.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/acard-ahci.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ahci_platform.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ata_generic.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/libahci_platform.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_acpi.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ali.ko 2024-03-16T09:43:52.577 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_amd.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_artop.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atiixp.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atp867x.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd640.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd64x.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cypress.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_efar.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt366.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt37x.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it8213.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it821x.ko 2024-03-16T09:43:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_jmicron.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_marvell.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_mpiix.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_netcell.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ninja32.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87410.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87415.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_oldpiix.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_opti.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_optidma.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pcmcia.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_piccolo.ko 2024-03-16T09:43:52.579 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_rdc.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sch.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_serverworks.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sil680.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sis.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sl82c105.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_triflex.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_via.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pdc_adma.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_inic162x.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_mv.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_nv.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_promise.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_qstor.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil.ko 2024-03-16T09:43:52.580 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil24.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sis.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_svw.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sx4.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_uli.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_via.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_vsc.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay/ks0108.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma 2024-03-16T09:43:52.581 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma/bcma.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe/aoe.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/brd.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd/drbd.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/floppy.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/loop.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/nbd.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk/null_blk.ko 2024-03-16T09:43:52.582 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/pktcdvd.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/rbd.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/virtio_blk.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkfront.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram/zram.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hangcheck-timer.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-16T09:43:52.583 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/via-rng.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/lp.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave/mwave.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ppdev.ko 2024-03-16T09:43:52.584 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tlclk.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/uv_mmtimer.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/virtio_console.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-16T09:43:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-aes.ko 2024-03-16T09:43:52.586 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-sha.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca/dca.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/amd64_edac.ko 2024-03-16T09:43:52.587 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/e752x_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/edac_mce_amd.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3000_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3200_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5100_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5400_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7300_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7core_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i82975x_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/ie31200_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/sb_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/x38_edac.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/edd.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-16T09:43:52.588 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-ich.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-appleir.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-aureal.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-axff.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-dr.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elecom.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elo.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-emsff.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gaff.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gt683r.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gyration.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-16T09:43:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtekff.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-hyperv.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-icade.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-keytouch.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-kye.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lcpower.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-led.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lenovo.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-multitouch.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-ortek.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-penmount.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-petalynx.ko 2024-03-16T09:43:52.590 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-picolcd.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-pl.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-plantronics.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-primax.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-rmi.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-common.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-16T09:43:52.591 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-saitek.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-samsung.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sjoy.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sony.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-speedlink.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-steelseries.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sunplus.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tivo.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tmff.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-topseed.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-twinhan.ko 2024-03-16T09:43:52.592 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-uclogic.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-waltop.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-wiimote.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-xinmo.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zpff.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zydacron.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/uhid.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/wacom.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_balloon.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_utils.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_vmbus.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru3.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-16T09:43:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7414.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7418.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adc128d818.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1025.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1026.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1029.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1031.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm9240.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ads7828.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7410.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7411.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7462.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7470.ko 2024-03-16T09:43:52.594 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7475.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7x10.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/amc6821.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/applesmc.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asb100.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asc7621.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/atxp1.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/coretemp.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dme1737.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds1621.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds620.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc1403.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc6w201.ko 2024-03-16T09:43:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71805f.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71882fg.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f75375s.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fam15h_power.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fschmd.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g760a.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g762.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl518sm.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl520sm.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5500_temp.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5k_amb.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmaem.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmpex.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina209.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina2xx.ko 2024-03-16T09:43:52.596 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/it87.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k10temp.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k8temp.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lineage-pem.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm63.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm73.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm75.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm77.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm78.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm80.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm83.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm85.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm87.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm90.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm92.ko 2024-03-16T09:43:52.597 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm93.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95234.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95241.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95245.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc2945.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4151.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4215.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4222.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4245.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4260.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4261.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max16065.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1619.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1668.ko 2024-03-16T09:43:52.598 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max197.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6639.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6650.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6697.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/mcp3021.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6683.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775-core.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct7802.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87360.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87427.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pcf8591.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-16T09:43:52.599 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/powr1220.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5627.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5636.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht15.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht21.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/shtc1.ko 2024-03-16T09:43:52.600 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sis5595.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47b397.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m1.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m192.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/thmc50.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp102.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp103.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp401.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp421.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via-cputemp.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via686a.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt1211.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt8231.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627ehf.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627hf.ko 2024-03-16T09:43:52.601 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83781d.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83791d.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83792d.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83793.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83795.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l785ts.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l786ng.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-16T09:43:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-16T09:43:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-dev.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-mux.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-smbus.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-stub.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_core.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-16T09:43:52.604 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-16T09:43:52.605 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/gameport.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/joydev.ko 2024-03-16T09:43:52.606 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/appletouch.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/bcm5974.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/cyapatp.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/sermouse.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/altera_ps2.ko 2024-03-16T09:43:52.607 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/arc_ps2.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/serio_raw.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/sparse-keymap.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-blinkm.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lm3530.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lp3944.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-ss4200.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-16T09:43:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache/bcache.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-bio-prison.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache-smq.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-crypt.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-delay.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-flakey.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-log-userspace.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-multipath.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-queue-length.ko 2024-03-16T09:43:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-raid.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-round-robin.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-service-time.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-switch.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-thin-pool.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-verity.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid0.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid1.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid10.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid456.ko 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message 2024-03-16T09:43:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptbase.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptctl.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptfc.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptlan.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptsas.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptscsih.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptspi.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_ich.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_sch.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/mfd-core.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/sm501.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/viperboard.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/vx855.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/wl1273-core.ko 2024-03-16T09:43:52.611 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds9802als.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds990x.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/bh1770glc.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710/cb710.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/at24.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/max6875.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/enclosure.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/hpilo.ko 2024-03-16T09:43:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29003.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29020.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-me.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-txe.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-16T09:43:52.613 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_7xx1.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_core.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tsl2550.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_balloon.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding/bonding.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-16T09:43:52.614 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dummy.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/eql.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-16T09:43:52.615 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros 2024-03-16T09:43:52.616 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-16T09:43:52.617 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-16T09:43:52.618 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-16T09:43:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dnet.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ethoc.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/fealnx.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-16T09:43:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-16T09:43:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/jme.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-16T09:43:52.622 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-16T09:43:52.623 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-16T09:43:52.624 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun 2024-03-16T09:43:52.625 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom 2024-03-16T09:43:52.626 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/geneve.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ifb.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvlan.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvtap.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-16T09:43:52.627 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mii.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/net_failover.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/netconsole.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/nlmon.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/amd.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm87xx.ko 2024-03-16T09:43:52.628 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/broadcom.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/cicada.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/davicom.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/dp83640.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/et1011c.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/icplus.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/lxt.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/marvell.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/micrel.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/national.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/phylink.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/qsemi.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/realtek.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/smsc.ko 2024-03-16T09:43:52.629 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/ste10Xp.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/vitesse.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_async.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppoe.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppox.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pptp.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slhc.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slip.ko 2024-03-16T09:43:52.630 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/sungem_phy.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tap.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_random.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tun.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/veth.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/virtio_net.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan/vxlan.ko 2024-03-16T09:43:52.631 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netfront.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb/ntb.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme-core.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_cs.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_pc.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_serial.ko 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci 2024-03-16T09:43:52.632 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/xen-pcifront.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/i82092.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/pd6729.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86 2024-03-16T09:43:52.633 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acerhdf.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/hdaps.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel_ips.ko 2024-03-16T09:43:52.634 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-16T09:43:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps_parport.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/pps_core.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp_kvm.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-16T09:43:52.636 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-em3027.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-max6900.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-16T09:43:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-x1205.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-9xxx.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-sas.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-xxxx.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/BusLogic.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/a100u2w.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/advansys.ko 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx 2024-03-16T09:43:52.638 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/am53c974.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/atp870u.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa/bfa.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ch.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-16T09:43:52.639 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dc395x.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dmx3191d.ko 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r 2024-03-16T09:43:52.640 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esp_scsi.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic/fnic.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hpsa.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hptiop.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hv_storvsc.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/initio.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ipr.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ips.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci/isci.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-16T09:43:52.641 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc/libfc.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas/libsas.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-16T09:43:52.642 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvumi.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pmcraid.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla1280.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/raid_class.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_debug.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ses.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/st.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/stex.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/virtio_scsi.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/wd719x.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb/ssb.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-16T09:43:52.644 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_file.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_iblock.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_mod.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_pscsi.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_user.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-16T09:43:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty 2024-03-16T09:43:52.646 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_gsm.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_hdlc.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/nozomi.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/arc_uart.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-16T09:43:52.647 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_aec.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_cif.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_pci_generic.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_sercos3.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-acm.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usblp.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usbtmc.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image 2024-03-16T09:43:52.648 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/mdc800.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/microtek.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/adutux.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/appledisplay.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi26.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi62.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ezusb.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/idmouse.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/iowarrior.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ldusb.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/legousbtower.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-16T09:43:52.649 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usb3503.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usblcd.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/uss720.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/yurex.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy/phy-generic.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/aircable.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ark3116.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ch341.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cp210x.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cyberjack.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-16T09:43:52.650 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/empeg.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_ti.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipaq.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipw.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ir-usb.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mct_u232.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7720.ko 2024-03-16T09:43:52.651 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7840.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/navman.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/omninet.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/opticon.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/option.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/oti6858.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/pl2303.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcaux.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcserial.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/quatech2.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/safe_serial.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/sierra.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ssu100.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/symbolserial.ko 2024-03-16T09:43:52.652 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_debug.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/visor.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/whiteheat.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/uas.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-16T09:43:52.653 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-karma.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/usb-storage.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio 2024-03-16T09:43:52.654 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_net.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_scsi.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/apple_bl.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/lcd.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio 2024-03-16T09:43:52.655 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_balloon.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_mmio.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_ring.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2482.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2490.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-16T09:43:52.656 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/wire.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/hpwdt.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/i6300esb.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-16T09:43:52.657 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ib700wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ibmasr.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it87_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/machzwd.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/nv_tco.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/softdog.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-16T09:43:52.658 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/via_wdt.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/wdt_pci.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/xen_wdt.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-evtchn.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntalloc.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntdev.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-privcmd.ko 2024-03-16T09:43:52.659 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p/9p.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs/affs.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs/befs.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/binfmt_misc.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs/btrfs.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles/cachefiles.ko 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph 2024-03-16T09:43:52.660 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph/ceph.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda/coda.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs/cramfs.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm/dlm.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs/f2fs.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/fat.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/msdos.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/vfat.ko 2024-03-16T09:43:52.661 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/cuse.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/fuse.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2/gfs2.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs/hfs.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus/hfsplus.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs/isofs.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs/jfs.ko 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd 2024-03-16T09:43:52.662 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd/lockd.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix/minix.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs/netfs.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfs.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv3.ko 2024-03-16T09:43:52.663 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv4.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/grace.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/nfs_acl.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd/nfsd.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2/nilfs2.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-celtic.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-centeuro.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-croatian.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-cyrillic.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-gaelic.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-greek.ko 2024-03-16T09:43:52.664 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-iceland.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-inuit.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-roman.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-romanian.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-turkish.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1250.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1251.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1255.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp737.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp775.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp850.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp852.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp855.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp857.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp860.ko 2024-03-16T09:43:52.665 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp861.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp862.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp863.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp864.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp865.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp866.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp869.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp874.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp932.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp936.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp949.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp950.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_euc-jp.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-1.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-13.ko 2024-03-16T09:43:52.666 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-14.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-15.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-2.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-3.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-4.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-5.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-6.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-7.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-9.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-r.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-ru.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-u.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_utf8.ko 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2 2024-03-16T09:43:52.667 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs/overlay.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore/ramoops.ko 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs 2024-03-16T09:43:52.668 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs/reiserfs.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs/romfs.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client/cifs.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_arc4.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_md4.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs/squashfs.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv/sysv.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf/udf.ko 2024-03-16T09:43:52.669 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs/ufs.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs/xfs.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu/rcutorture.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/torture.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/asn1_encoder.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc-itu-t.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc8.ko 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto 2024-03-16T09:43:52.670 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libarc4.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libdes.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/libcrc32c.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lru_cache.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4_compress.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4hc_compress.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math/cordic.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6/raid6_pq.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_bm.ko 2024-03-16T09:43:52.671 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_fsm.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_kmp.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd/zstd_compress.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm/hwpoison-inject.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/6lowpan.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_dest.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_fragment.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_hop.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_mobility.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_routing.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_udp.ko 2024-03-16T09:43:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/garp.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/mrp.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/p8022.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/psnap.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/stp.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q/8021q.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_fd.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_rdma.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_virtio.ko 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv 2024-03-16T09:43:52.673 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv/batman-adv.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/br_netfilter.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/bridge.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-16T09:43:52.674 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtables.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph/libceph.ko 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core 2024-03-16T09:43:52.675 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/failover.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/pktgen.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_diag.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv4.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv6.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver/dns_resolver.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/dsa_core.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_brcm.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_dsa.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_none.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_trailer.ko 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan 2024-03-16T09:43:52.676 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ah4.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/esp4.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/fou.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/gre.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/inet_diag.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_gre.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_tunnel.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_vti.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipcomp.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipip.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-16T09:43:52.677 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-16T09:43:52.678 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_bic.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_dctcp.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_diag.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_highspeed.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_htcp.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_hybla.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_illinois.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_lp.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_scalable.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_vegas.ko 2024-03-16T09:43:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_veno.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_westwood.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_yeah.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tunnel4.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_diag.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_tunnel.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ah6.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/esp6.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/fou6.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_tunnel.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_vti.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ipcomp6.ko 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter 2024-03-16T09:43:52.680 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-16T09:43:52.681 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/sit.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/tunnel6.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key/af_key.ko 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp 2024-03-16T09:43:52.682 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_core.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_eth.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip6.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_netlink.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ppp.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc/llc.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154/mac802154.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls/mpls_gso.ko 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset 2024-03-16T09:43:52.683 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-16T09:43:52.684 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-16T09:43:52.685 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conncount.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-16T09:43:52.686 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_log_syslog.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_irc.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_sip.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_tables.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink.ko 2024-03-16T09:43:52.687 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_log.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_chain_nat.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_compat.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_ct.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fib.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_hash.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_limit.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_log.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_masq.ko 2024-03-16T09:43:52.688 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_nat.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_numgen.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_queue.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_quota.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_redir.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject_inet.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_socket.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_tunnel.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_AUDIT.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CT.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_DSCP.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HL.ko 2024-03-16T09:43:52.689 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HMARK.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LED.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LOG.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NETMAP.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFLOG.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_RATEEST.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_SECMARK.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TEE.ko 2024-03-16T09:43:52.690 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TPROXY.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TRACE.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_addrtype.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_bpf.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cgroup.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cluster.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_comment.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connbytes.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlabel.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlimit.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connmark.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_conntrack.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cpu.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dccp.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_devgroup.ko 2024-03-16T09:43:52.691 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dscp.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ecn.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_esp.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hashlimit.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_helper.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hl.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipcomp.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_iprange.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipvs.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_l2tp.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_length.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_limit.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mac.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mark.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_multiport.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nat.ko 2024-03-16T09:43:52.692 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nfacct.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_osf.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_owner.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_physdev.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_pkttype.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_policy.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_quota.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_rateest.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_realm.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_recent.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_sctp.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_set.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_socket.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_state.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_statistic.ko 2024-03-16T09:43:52.693 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_string.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_tcpmss.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_time.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_u32.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink/netlink_diag.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh/nsh.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/openvswitch.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-geneve.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-gre.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-vxlan.ko 2024-03-16T09:43:52.694 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet/af_packet_diag.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_rdma.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_tcp.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_csum.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_gact.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_mirred.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_nat.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_pedit.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_police.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_simple.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_skbedit.ko 2024-03-16T09:43:52.695 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_vlan.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_basic.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_bpf.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_flow.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_fw.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_route.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_u32.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_cmp.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_ipset.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_meta.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_nbyte.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_text.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_u32.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_choke.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_codel.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_drr.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_fq.ko 2024-03-16T09:43:52.696 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_gred.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hfsc.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hhf.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_htb.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_ingress.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio_lib.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_multiq.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_netem.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_pie.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_plug.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_prio.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_qfq.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_red.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfb.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfq.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_tbf.ko 2024-03-16T09:43:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_teql.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp_diag.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/sunrpc.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/diag.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/tipc.ko 2024-03-16T09:43:52.698 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix/unix_diag.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys 2024-03-16T09:43:52.699 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys/trusted.ko 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib/irqbypass.ko 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias.bin 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.alias.bin 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.bin 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.modinfo 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep.bin 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.devname 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.order 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.softdep 2024-03-16T09:43:52.700 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols 2024-03-16T09:43:52.701 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols.bin 2024-03-16T09:43:52.701 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T09:43:52.705 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:52.705 DEBUG:teuthology.orchestra.run.smithi039:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout:postinstall scriptlet (using /bin/sh): 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout: /usr/bin/kernel-install add 6.8.0-rc7-g3845b7ac715a /lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout:for file in vmlinuz System.map config; do 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a"; then 2024-03-16T09:43:52.728 INFO:teuthology.orchestra.run.smithi039.stdout: cp "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a" 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout: fi 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:done 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:new-kernel-pkg --remove 6.8.0-rc7-g3845b7ac715a --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-g3845b7ac715a.img 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-install remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:52.729 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-03-16T09:43:52.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.06 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 68.el9 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Group : Unspecified 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 8264254 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T09:43:52.810 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : builder@centos.org 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : CentOS 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:43:52.811 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T09:43:52.812 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T09:43:52.812 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:52.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T09:43:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-03-16T09:43:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T09:43:54.786 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-03-16T09:43:54.788 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-03-16T09:43:54.803 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.NY5oBS1mUa 2024-03-16T09:43:54.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NY5oBS1mUa 2024-03-16T09:43:54.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.NY5oBS1mUa 2024-03-16T09:43:55.006 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.NY5oBS1mUa is 6KB 2024-03-16T09:43:55.054 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.NY5oBS1mUa 2024-03-16T09:43:55.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T09:43:55.138 INFO:teuthology.orchestra.run.smithi039.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T09:43:55.138 INFO:teuthology.orchestra.run.smithi039.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T09:43:55.138 INFO:teuthology.orchestra.run.smithi039.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T09:43:55.138 INFO:teuthology.orchestra.run.smithi039.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a.conf 2024-03-16T09:43:55.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a 2024-03-16T09:43:55.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-03-16T09:44:25.296 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-16T09:44:25.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:44:25.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:44:43.806 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-16T09:44:52.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:44:52.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:44:55.874 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-16T09:45:07.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:45:07.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:45:10.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-16T09:45:25.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:45:25.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:46:25.954 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T09:46:43.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:46:43.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:46:44.434 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-03-16T09:46:45.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:46:45.128 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-g3845b7ac715a"... 2024-03-16T09:46:45.128 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-03-16T09:46:45.184 INFO:teuthology.orchestra.run.smithi039.stdout:6.8.0-rc7-g3845b7ac715a 2024-03-16T09:46:45.184 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-g3845b7ac715a vs 6.8.0-rc7-g3845b7ac715a 2024-03-16T09:46:45.185 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T09:46:45.185 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T09:46:46.186 DEBUG:teuthology.parallel:result is None 2024-03-16T09:46:46.186 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T09:46:46.194 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T09:46:46.194 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T09:46:46.202 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T09:46:46.204 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T09:46:46.253 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T09:46:46.260 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T09:46:46.272 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T09:46:46.272 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T09:46:46.278 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T09:46:46.280 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T09:46:46.389 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T09:46:46.396 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T09:46:46.396 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T09:46:46.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:46:46.412 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T09:46:46.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:46:46.428 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T09:46:46.436 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T09:46:46.454 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T09:46:46.502 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.511 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.521 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.530 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.561 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.625 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T09:46:46.630 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T09:46:46.637 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T09:46:46.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T09:46:46.644 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T09:46:46.647 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T09:46:46.739 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T09:46:46.749 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T09:46:46.749 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T09:46:46.756 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T09:46:46.758 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T09:46:46.805 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:46.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:47.007 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.203 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:46:47.203 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T09:46:47.297 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:47.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:47.408 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.448 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.495 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:46:47.495 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T09:46:47.563 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:47.608 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T09:46:47.664 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.749 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T09:46:47.794 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:46:47.794 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T09:46:47.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-03-16T09:46:47.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-16T09:46:47.874 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-03-16T09:46:47.922 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:46:47.944 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:46:47.998 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:46:48.392 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T09:46:48.400 INFO:teuthology.task.internal:Starting timer... 2024-03-16T09:46:48.400 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T09:46:48.410 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T09:46:48.418 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-16T09:46:48.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-03-16T09:46:48.558 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-03-16T09:46:48.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-16T09:46:48.621 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-16T09:46:48.623 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-03-16T09:46:48.680 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-03-16T09:46:48.682 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T09:46:48.683 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-03-16T09:46:48.748 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-03-16T09:46:48.748 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-16T09:46:48.769 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-16T09:46:48.770 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-03-16T09:46:48.791 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-03-16T09:46:48.791 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-03-16T09:46:48.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:46:48.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:46:48.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:46:48.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:46:48.901 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:46:48.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:46:48.930 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T09:46:48.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-03-16T09:46:49.046 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-16T09:46:49.079 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-03-16T09:46:49.117 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T09:46:49.130 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-16T09:46:49.137 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-16T09:46:49.258 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T09:46:49.273 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T09:46:49.274 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi105.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T09:52:52.938 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-03-16T09:52:52.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:52:52.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:52:53.091 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-03-16T09:52:53.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-16T09:52:53.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-16T09:52:53.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:52:53.332 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-16T09:52:53.401 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-16T09:52:53.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-16T09:52:53.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:52:53.486 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-03-16T09:52:53.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-16T09:52:53.559 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T09:52:53.571 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T09:52:53.571 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:52:53.572 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:52:53.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:52:53.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:52:53.582 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:52:53.582 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:52:53.611 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:52:53.628 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:52:53.632 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:52:53.649 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:52:53.658 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-16T09:52:53.672 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-16T09:52:53.678 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-03-16T09:52:53.683 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:52:53.688 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:52:53.692 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-03-16T09:52:53.704 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:52:53.708 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:52:53.724 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:52:53.740 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:52:53.748 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:52:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:52:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-16T09:52:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.753 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.768 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:52:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:52:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-16T09:52:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.772 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-03-16T09:52:53.893 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-03-16T09:52:53.947 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:52:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:52:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:52:54.143 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:52:54.143 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-03-16T09:52:54.143 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:54.144 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:54.144 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:54.144 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:52:54.145 INFO:teuthology.run_tasks:Running task install... 2024-03-16T09:52:54.154 DEBUG:teuthology.task.install:project ceph 2024-03-16T09:52:54.154 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T09:52:54.154 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T09:52:54.154 INFO:teuthology.task.install:Using flavor: default 2024-03-16T09:52:54.160 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T09:52:54.160 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T09:52:54.161 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:52:54.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:52:54.163 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:52:54.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:52:54.164 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:52:54.165 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:52:54.310 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:52:54.310 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:52:54.381 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:52:54.381 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:52:54.385 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:52:54.385 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:52:54.484 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:52:54.485 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:52:54.485 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:52:54.513 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:52:54.513 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:52:54.549 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:52:54.549 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:52:54.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:52:54.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:52:54.557 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:52:54.557 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:52:54.582 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:52:54.589 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:52:54.590 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:52:54.657 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:52:54.657 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:52:54.660 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:52:54.662 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:52:54.728 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-03-16T09:52:54.730 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-03-16T09:52:54.742 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:52:54.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:52:54.809 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-16T09:52:54.811 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-16T09:52:54.932 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:52:54.965 INFO:teuthology.orchestra.run.smithi194.stdout:74 files removed 2024-03-16T09:52:54.993 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:52:55.027 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-03-16T09:52:55.032 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-03-16T09:52:55.044 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-16T09:52:55.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:52:55.392 INFO:teuthology.orchestra.run.smithi039.stdout:74 files removed 2024-03-16T09:52:55.431 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:52:55.677 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 211 kB/s | 83 kB 00:00 2024-03-16T09:52:55.678 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 262 kB/s | 83 kB 00:00 2024-03-16T09:52:56.094 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-16T09:52:56.112 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-16T09:52:56.397 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 184 kB/s | 83 kB 00:00 2024-03-16T09:52:56.428 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-03-16T09:52:56.471 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-03-16T09:52:56.936 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-16T09:52:56.973 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-16T09:52:57.198 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - AppStream 9.0 MB/s | 6.7 MB 00:00 2024-03-16T09:52:57.541 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-16T09:52:58.144 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-16T09:52:58.424 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-03-16T09:52:58.623 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-03-16T09:52:59.207 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-16T09:52:59.430 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-16T09:52:59.928 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-16T09:53:00.966 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-16T09:53:03.151 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-16T09:53:05.024 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-03-16T09:53:08.232 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.1 MB 00:01 2024-03-16T09:53:09.875 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.1 MB 00:01 2024-03-16T09:53:10.002 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-16T09:53:11.217 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 15 kB 00:00 2024-03-16T09:53:11.321 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-03-16T09:53:12.236 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-16T09:53:17.219 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-03-16T09:53:17.320 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-16T09:53:19.683 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:53:19.684 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:53:19.698 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:53:19.699 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:53:19.700 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:53:19.702 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:53:19.704 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:53:19.707 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:53:19.708 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:53:19.788 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:53:19.802 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:53:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:53:19.804 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:53:19.805 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:53:19.806 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:53:19.807 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:53:19.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:53:19.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:53:19.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:53:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-16T09:53:19.812 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:53:19.813 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 224 M 2024-03-16T09:53:19.813 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-16T09:53:20.186 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:53:20.188 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:53:20.210 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:53:20.212 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:53:20.213 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:53:20.215 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:53:20.218 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:53:20.222 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:53:20.321 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-03-16T09:53:20.347 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================ 2024-03-16T09:53:20.347 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-03-16T09:53:20.347 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================ 2024-03-16T09:53:20.347 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:53:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:53:20.349 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-03-16T09:53:20.350 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:53:20.351 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:53:20.352 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:53:20.353 INFO:teuthology.orchestra.run.smithi039.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:53:20.354 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:53:20.355 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:53:20.356 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:53:20.357 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:53:20.358 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:53:20.359 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:53:20.360 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================================ 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout:Install 150 Packages 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 24 Packages 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:53:20.361 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 224 M 2024-03-16T09:53:20.362 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-03-16T09:53:20.378 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-03-16T09:53:20.604 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 3.9 MB/s | 890 kB 00:00 2024-03-16T09:53:20.654 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 2.8 MB/s | 145 kB 00:00 2024-03-16T09:53:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 6.7 MB/s | 2.4 MB 00:00 2024-03-16T09:53:21.047 INFO:teuthology.orchestra.run.smithi039.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-03-16T09:53:21.071 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-03-16T09:53:21.280 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-03-16T09:53:21.714 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 7.3 MB/s | 4.7 MB 00:00 2024-03-16T09:53:21.714 INFO:teuthology.orchestra.run.smithi039.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.3 MB/s | 890 kB 00:00 2024-03-16T09:53:22.173 INFO:teuthology.orchestra.run.smithi039.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 316 kB/s | 145 kB 00:00 2024-03-16T09:53:22.606 INFO:teuthology.orchestra.run.smithi039.stdout:(4/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 3.0 MB/s | 5.4 MB 00:01 2024-03-16T09:53:22.940 INFO:teuthology.orchestra.run.smithi039.stdout:(5/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-03-16T09:53:23.080 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 7.2 MB/s | 21 MB 00:02 2024-03-16T09:53:23.197 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 6.6 MB/s | 9.8 MB 00:01 2024-03-16T09:53:23.222 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el 177 kB/s | 25 kB 00:00 2024-03-16T09:53:23.325 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 2.2 MB/s | 221 kB 00:00 2024-03-16T09:53:23.398 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 360 kB/s | 27 kB 00:00 2024-03-16T09:53:23.448 INFO:teuthology.orchestra.run.smithi039.stdout:(6/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 9.3 MB/s | 4.7 MB 00:00 2024-03-16T09:53:23.465 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 475 kB/s | 32 kB 00:00 2024-03-16T09:53:23.732 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 2.7 MB/s | 747 kB 00:00 2024-03-16T09:53:23.791 INFO:teuthology.orchestra.run.smithi039.stdout:(7/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-03-16T09:53:23.854 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 2.4 MB/s | 163 kB 00:00 2024-03-16T09:53:23.958 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 6.3 MB/s | 17 MB 00:02 2024-03-16T09:53:23.974 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): librados-devel-19.0.0-2162.g3d78f5ba. 723 kB/s | 127 kB 00:00 2024-03-16T09:53:24.100 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): libradosstriper1-19.0.0-2162.g3d78f5b 3.5 MB/s | 501 kB 00:00 2024-03-16T09:53:24.183 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 543 kB/s | 45 kB 00:00 2024-03-16T09:53:24.267 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.5 MB/s | 131 kB 00:00 2024-03-16T09:53:24.350 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.9 MB/s | 162 kB 00:00 2024-03-16T09:53:24.442 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.4 MB/s | 320 kB 00:00 2024-03-16T09:53:24.534 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.2 MB/s | 298 kB 00:00 2024-03-16T09:53:24.617 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.2 MB/s | 99 kB 00:00 2024-03-16T09:53:24.701 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-16T09:53:25.168 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-03-16T09:53:25.260 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 5.5 MB/s | 3.1 MB 00:00 2024-03-16T09:53:25.285 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 1.4 MB/s | 170 kB 00:00 2024-03-16T09:53:25.311 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2162.g 677 kB/s | 34 kB 00:00 2024-03-16T09:53:25.344 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 2.3 MB/s | 139 kB 00:00 2024-03-16T09:53:25.415 INFO:teuthology.orchestra.run.smithi039.stdout:(8/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x86 8.6 MB/s | 17 MB 00:01 2024-03-16T09:53:25.499 INFO:teuthology.orchestra.run.smithi039.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 306 kB/s | 25 kB 00:00 2024-03-16T09:53:25.553 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 7.4 MB/s | 1.8 MB 00:00 2024-03-16T09:53:25.645 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.6 MB/s | 248 kB 00:00 2024-03-16T09:53:25.737 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 540 kB/s | 49 kB 00:00 2024-03-16T09:53:25.812 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 219 kB/s | 16 kB 00:00 2024-03-16T09:53:25.904 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 2.8 MB/s | 264 kB 00:00 2024-03-16T09:53:26.088 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 4.0 MB/s | 748 kB 00:00 2024-03-16T09:53:26.171 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 219 kB/s | 18 kB 00:00 2024-03-16T09:53:26.305 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-16T09:53:26.356 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-16T09:53:26.389 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-16T09:53:26.431 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-16T09:53:26.464 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 951 kB/s | 32 kB 00:00 2024-03-16T09:53:26.573 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-16T09:53:26.607 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 517 kB/s | 17 kB 00:00 2024-03-16T09:53:26.641 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-16T09:53:26.674 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-16T09:53:26.766 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-03-16T09:53:26.791 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 377 kB/s | 44 kB 00:00 2024-03-16T09:53:26.825 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-16T09:53:26.859 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-16T09:53:26.884 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 804 kB/s | 46 kB 00:00 2024-03-16T09:53:26.917 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-16T09:53:26.943 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-03-16T09:53:26.968 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-16T09:53:26.994 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-16T09:53:27.085 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-03-16T09:53:27.127 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-03-16T09:53:27.153 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-16T09:53:27.178 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-16T09:53:27.303 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-16T09:53:27.337 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-16T09:53:27.362 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-16T09:53:27.488 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2024-03-16T09:53:27.509 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 169 kB/s | 24 kB 00:00 2024-03-16T09:53:27.534 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 389 kB/s | 18 kB 00:00 2024-03-16T09:53:27.560 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-16T09:53:27.585 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-16T09:53:27.686 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-16T09:53:27.711 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 217 kB 00:00 2024-03-16T09:53:27.744 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-03-16T09:53:27.770 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-03-16T09:53:27.795 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-03-16T09:53:27.820 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-16T09:53:27.846 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-16T09:53:27.871 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-16T09:53:27.896 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-03-16T09:53:27.921 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-16T09:53:28.072 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 261 kB/s | 39 kB 00:00 2024-03-16T09:53:28.232 INFO:teuthology.orchestra.run.smithi039.stdout:(10/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el 2.2 MB/s | 9.8 MB 00:04 2024-03-16T09:53:28.281 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-16T09:53:28.306 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 177 kB/s | 41 kB 00:00 2024-03-16T09:53:28.348 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-16T09:53:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 842 kB/s | 56 kB 00:00 2024-03-16T09:53:28.383 INFO:teuthology.orchestra.run.smithi039.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 1.5 MB/s | 221 kB 00:00 2024-03-16T09:53:28.398 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-16T09:53:28.440 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 656 kB/s | 44 kB 00:00 2024-03-16T09:53:28.449 INFO:teuthology.orchestra.run.smithi039.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 412 kB/s | 27 kB 00:00 2024-03-16T09:53:28.465 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-16T09:53:28.491 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T09:53:28.541 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-16T09:53:28.566 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-16T09:53:28.591 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-16T09:53:28.617 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-16T09:53:28.733 INFO:teuthology.orchestra.run.smithi039.stdout:(13/174): ceph-common-19.0.0-2162.g3d78f5ba.el9 2.6 MB/s | 21 MB 00:07 2024-03-16T09:53:28.742 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-16T09:53:28.758 INFO:teuthology.orchestra.run.smithi039.stdout:(14/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 103 kB/s | 32 kB 00:00 2024-03-16T09:53:28.793 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-03-16T09:53:28.818 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-03-16T09:53:28.843 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-16T09:53:28.859 INFO:teuthology.orchestra.run.smithi039.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.6 MB/s | 163 kB 00:00 2024-03-16T09:53:28.968 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-03-16T09:53:28.985 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-16T09:53:28.992 INFO:teuthology.orchestra.run.smithi039.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 958 kB/s | 127 kB 00:00 2024-03-16T09:53:29.043 INFO:teuthology.orchestra.run.smithi039.stdout:(17/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 2.4 MB/s | 747 kB 00:00 2024-03-16T09:53:29.519 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 7.5 MB/s | 48 MB 00:06 2024-03-16T09:53:29.544 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 439 kB/s | 245 kB 00:00 2024-03-16T09:53:29.653 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.0 MB/s | 216 kB 00:00 2024-03-16T09:53:29.660 INFO:teuthology.orchestra.run.smithi039.stdout:(18/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 11 MB/s | 48 MB 00:04 2024-03-16T09:53:29.693 INFO:teuthology.orchestra.run.smithi039.stdout:(19/174): libradosstriper1-19.0.0-2162.g3d78f5b 716 kB/s | 501 kB 00:00 2024-03-16T09:53:29.711 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 599 kB/s | 35 kB 00:00 2024-03-16T09:53:29.718 INFO:teuthology.orchestra.run.smithi039.stdout:(20/174): python3-ceph-argparse-19.0.0-2162.g3d 788 kB/s | 45 kB 00:00 2024-03-16T09:53:29.745 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.1 MB/s | 248 kB 00:00 2024-03-16T09:53:29.769 INFO:teuthology.orchestra.run.smithi039.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 3.2 MB/s | 162 kB 00:00 2024-03-16T09:53:29.770 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 759 kB/s | 44 kB 00:00 2024-03-16T09:53:29.794 INFO:teuthology.orchestra.run.smithi039.stdout:(22/174): python3-ceph-common-19.0.0-2162.g3d78 1.3 MB/s | 131 kB 00:00 2024-03-16T09:53:29.804 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-16T09:53:29.828 INFO:teuthology.orchestra.run.smithi039.stdout:(23/174): python3-rados-19.0.0-2162.g3d78f5ba.e 5.4 MB/s | 320 kB 00:00 2024-03-16T09:53:29.878 INFO:teuthology.orchestra.run.smithi039.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 2.0 MB/s | 99 kB 00:00 2024-03-16T09:53:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:(25/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 2.5 MB/s | 298 kB 00:00 2024-03-16T09:53:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-16T09:53:29.954 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.2 MB/s | 4.1 MB 00:00 2024-03-16T09:53:30.013 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-03-16T09:53:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-16T09:53:30.121 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-03-16T09:53:30.154 INFO:teuthology.orchestra.run.smithi039.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 293 kB/s | 34 kB 00:00 2024-03-16T09:53:30.271 INFO:teuthology.orchestra.run.smithi039.stdout:(29/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 8.6 MB/s | 3.1 MB 00:00 2024-03-16T09:53:30.296 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-03-16T09:53:30.322 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-16T09:53:30.355 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 903 kB/s | 309 kB 00:00 2024-03-16T09:53:30.380 INFO:teuthology.orchestra.run.smithi039.stdout:(30/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 3.8 MB/s | 5.1 MB 00:01 2024-03-16T09:53:30.397 INFO:teuthology.orchestra.run.smithi039.stdout:(31/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 574 kB/s | 139 kB 00:00 2024-03-16T09:53:30.398 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-03-16T09:53:30.448 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-03-16T09:53:30.481 INFO:teuthology.orchestra.run.smithi039.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 8.6 MB/s | 1.8 MB 00:00 2024-03-16T09:53:30.481 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-03-16T09:53:30.506 INFO:teuthology.orchestra.run.smithi039.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.2 MB/s | 248 kB 00:00 2024-03-16T09:53:30.523 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-16T09:53:30.531 INFO:teuthology.orchestra.run.smithi039.stdout:(34/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 1.0 MB/s | 49 kB 00:00 2024-03-16T09:53:30.556 INFO:teuthology.orchestra.run.smithi039.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2162.g3 334 kB/s | 16 kB 00:00 2024-03-16T09:53:30.582 INFO:teuthology.orchestra.run.smithi039.stdout:(36/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 5.2 MB/s | 264 kB 00:00 2024-03-16T09:53:30.616 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-16T09:53:30.632 INFO:teuthology.orchestra.run.smithi039.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 372 kB/s | 18 kB 00:00 2024-03-16T09:53:30.780 INFO:teuthology.orchestra.run.smithi039.stdout:(38/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 3.3 MB/s | 748 kB 00:00 2024-03-16T09:53:30.782 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-03-16T09:53:30.891 INFO:teuthology.orchestra.run.smithi039.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 920 kB 00:00 2024-03-16T09:53:30.892 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 157 kB/s | 43 kB 00:00 2024-03-16T09:53:30.918 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 238 kB/s | 32 kB 00:00 2024-03-16T09:53:30.925 INFO:teuthology.orchestra.run.smithi039.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-16T09:53:30.950 INFO:teuthology.orchestra.run.smithi039.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-16T09:53:30.983 INFO:teuthology.orchestra.run.smithi039.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-16T09:53:31.001 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-03-16T09:53:31.009 INFO:teuthology.orchestra.run.smithi039.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 573 kB/s | 32 kB 00:00 2024-03-16T09:53:31.027 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-03-16T09:53:31.052 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-16T09:53:31.085 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-16T09:53:31.109 INFO:teuthology.orchestra.run.smithi039.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-16T09:53:31.119 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-16T09:53:31.134 INFO:teuthology.orchestra.run.smithi039.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 17 kB 00:00 2024-03-16T09:53:31.144 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 383 kB/s | 35 kB 00:00 2024-03-16T09:53:31.176 INFO:teuthology.orchestra.run.smithi039.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 250 kB/s | 10 kB 00:00 2024-03-16T09:53:31.193 INFO:teuthology.orchestra.run.smithi039.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-16T09:53:31.243 INFO:teuthology.orchestra.run.smithi039.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 44 kB 00:00 2024-03-16T09:53:31.269 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-16T09:53:31.277 INFO:teuthology.orchestra.run.smithi039.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2024-03-16T09:53:31.295 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 961 kB/s | 144 kB 00:00 2024-03-16T09:53:31.328 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-03-16T09:53:31.341 INFO:teuthology.orchestra.run.smithi039.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 754 kB/s | 46 kB 00:00 2024-03-16T09:53:31.353 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-03-16T09:53:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-03-16T09:53:31.387 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-03-16T09:53:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 282 kB 00:00 2024-03-16T09:53:31.429 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-google-auth-2.28.0-1.el9.noa 590 kB/s | 203 kB 00:00 2024-03-16T09:53:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.3 MB/s | 187 kB 00:00 2024-03-16T09:53:31.446 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-03-16T09:53:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.4 MB/s | 217 kB 00:00 2024-03-16T09:53:31.471 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-03-16T09:53:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 717 kB/s | 34 kB 00:00 2024-03-16T09:53:31.555 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-03-16T09:53:31.580 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-03-16T09:53:31.638 INFO:teuthology.orchestra.run.smithi039.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-16T09:53:31.647 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-03-16T09:53:31.672 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-16T09:53:31.696 INFO:teuthology.orchestra.run.smithi039.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 8.3 MB/s | 1.5 MB 00:00 2024-03-16T09:53:31.706 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-03-16T09:53:31.731 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-16T09:53:31.746 INFO:teuthology.orchestra.run.smithi039.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.3 MB/s | 811 kB 00:00 2024-03-16T09:53:31.757 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-16T09:53:31.772 INFO:teuthology.orchestra.run.smithi039.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2024-03-16T09:53:31.790 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-03-16T09:53:31.824 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-16T09:53:31.941 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-03-16T09:53:31.966 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 252 kB/s | 36 kB 00:00 2024-03-16T09:53:31.972 INFO:teuthology.orchestra.run.smithi039.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 25 MB/s | 5.7 MB 00:00 2024-03-16T09:53:31.991 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 804 kB/s | 188 kB 00:00 2024-03-16T09:53:32.006 INFO:teuthology.orchestra.run.smithi039.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-03-16T09:53:32.042 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 506 kB/s | 51 kB 00:00 2024-03-16T09:53:32.067 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-16T09:53:32.081 INFO:teuthology.orchestra.run.smithi039.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 78 kB 00:00 2024-03-16T09:53:32.092 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 889 kB/s | 90 kB 00:00 2024-03-16T09:53:32.126 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-16T09:53:32.151 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-03-16T09:53:32.173 INFO:teuthology.orchestra.run.smithi039.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-03-16T09:53:32.176 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-16T09:53:32.198 INFO:teuthology.orchestra.run.smithi039.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 221 kB/s | 24 kB 00:00 2024-03-16T09:53:32.224 INFO:teuthology.orchestra.run.smithi039.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 367 kB/s | 18 kB 00:00 2024-03-16T09:53:32.249 INFO:teuthology.orchestra.run.smithi039.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-03-16T09:53:32.274 INFO:teuthology.orchestra.run.smithi039.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2024-03-16T09:53:32.343 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-03-16T09:53:32.425 INFO:teuthology.orchestra.run.smithi039.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-03-16T09:53:32.450 INFO:teuthology.orchestra.run.smithi039.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-16T09:53:32.460 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2024-03-16T09:53:32.519 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): librados2-19.0.0-2162.g3d78f5ba.el9. 9.2 MB/s | 3.4 MB 00:00 2024-03-16T09:53:32.592 INFO:teuthology.orchestra.run.smithi039.stdout:(70/174): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-03-16T09:53:32.625 INFO:teuthology.orchestra.run.smithi039.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-16T09:53:32.650 INFO:teuthology.orchestra.run.smithi039.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 345 kB/s | 69 kB 00:00 2024-03-16T09:53:32.669 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 6.4 MB/s | 3.1 MB 00:00 2024-03-16T09:53:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 41 kB 00:00 2024-03-16T09:53:32.701 INFO:teuthology.orchestra.run.smithi039.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-16T09:53:32.735 INFO:teuthology.orchestra.run.smithi039.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-16T09:53:32.760 INFO:teuthology.orchestra.run.smithi039.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-16T09:53:32.786 INFO:teuthology.orchestra.run.smithi039.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 596 kB/s | 47 kB 00:00 2024-03-16T09:53:32.786 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.0 MB/s | 2.3 MB 00:00 2024-03-16T09:53:32.811 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 299 kB/s | 42 kB 00:00 2024-03-16T09:53:32.837 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-03-16T09:53:32.878 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 5.1 MB/s | 1.8 MB 00:00 2024-03-16T09:53:32.886 INFO:teuthology.orchestra.run.smithi039.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 346 kB/s | 43 kB 00:00 2024-03-16T09:53:32.904 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 774 kB/s | 71 kB 00:00 2024-03-16T09:53:32.929 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 395 kB/s | 36 kB 00:00 2024-03-16T09:53:32.954 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-16T09:53:32.980 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 458 kB/s | 23 kB 00:00 2024-03-16T09:53:33.005 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-16T09:53:33.030 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-16T09:53:33.072 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-16T09:53:33.097 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-03-16T09:53:33.123 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-16T09:53:33.148 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 544 kB/s | 64 kB 00:00 2024-03-16T09:53:33.153 INFO:teuthology.orchestra.run.smithi039.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-16T09:53:33.178 INFO:teuthology.orchestra.run.smithi039.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 105 kB/s | 39 kB 00:00 2024-03-16T09:53:33.190 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-16T09:53:33.211 INFO:teuthology.orchestra.run.smithi039.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 130 kB/s | 41 kB 00:00 2024-03-16T09:53:33.232 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 20 MB/s | 2.2 MB 00:00 2024-03-16T09:53:33.253 INFO:teuthology.orchestra.run.smithi039.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-16T09:53:33.265 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-16T09:53:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 964 kB/s | 63 kB 00:00 2024-03-16T09:53:33.282 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-03-16T09:53:33.312 INFO:teuthology.orchestra.run.smithi039.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 300 kB 00:00 2024-03-16T09:53:33.324 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-16T09:53:33.337 INFO:teuthology.orchestra.run.smithi039.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.4 MB/s | 79 kB 00:00 2024-03-16T09:53:33.358 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-16T09:53:33.363 INFO:teuthology.orchestra.run.smithi039.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 547 kB/s | 44 kB 00:00 2024-03-16T09:53:33.388 INFO:teuthology.orchestra.run.smithi039.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-16T09:53:33.399 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 636 kB/s | 74 kB 00:00 2024-03-16T09:53:33.404 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:53:33.405 INFO:teuthology.orchestra.run.smithi105.stdout:Total 17 MB/s | 224 MB 00:13 2024-03-16T09:53:33.438 INFO:teuthology.orchestra.run.smithi039.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-16T09:53:33.463 INFO:teuthology.orchestra.run.smithi039.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2024-03-16T09:53:33.497 INFO:teuthology.orchestra.run.smithi039.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-16T09:53:33.522 INFO:teuthology.orchestra.run.smithi039.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-16T09:53:33.664 INFO:teuthology.orchestra.run.smithi039.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 949 kB/s | 187 kB 00:00 2024-03-16T09:53:33.689 INFO:teuthology.orchestra.run.smithi039.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-16T09:53:33.723 INFO:teuthology.orchestra.run.smithi039.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-16T09:53:33.765 INFO:teuthology.orchestra.run.smithi039.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-03-16T09:53:33.870 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-16T09:53:34.140 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-16T09:53:34.140 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-16T09:53:34.332 INFO:teuthology.orchestra.run.smithi039.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 33 kB/s | 21 kB 00:00 2024-03-16T09:53:34.482 INFO:teuthology.orchestra.run.smithi039.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 34 kB/s | 25 kB 00:00 2024-03-16T09:53:35.558 INFO:teuthology.orchestra.run.smithi039.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 200 kB/s | 245 kB 00:01 2024-03-16T09:53:35.641 INFO:teuthology.orchestra.run.smithi039.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 214 kB/s | 248 kB 00:01 2024-03-16T09:53:35.873 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-16T09:53:35.873 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-16T09:53:36.275 INFO:teuthology.orchestra.run.smithi039.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 55 kB/s | 35 kB 00:00 2024-03-16T09:53:36.526 INFO:teuthology.orchestra.run.smithi039.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 224 kB/s | 216 kB 00:00 2024-03-16T09:53:36.975 INFO:teuthology.orchestra.run.smithi039.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 64 kB/s | 44 kB 00:00 2024-03-16T09:53:37.059 INFO:teuthology.orchestra.run.smithi039.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 273 kB/s | 22 kB 00:00 2024-03-16T09:53:37.209 INFO:teuthology.orchestra.run.smithi039.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-03-16T09:53:37.360 INFO:teuthology.orchestra.run.smithi039.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 120 kB/s | 94 kB 00:00 2024-03-16T09:53:37.735 INFO:teuthology.orchestra.run.smithi039.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-16T09:53:37.810 INFO:teuthology.orchestra.run.smithi039.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 263 kB/s | 19 kB 00:00 2024-03-16T09:53:37.844 INFO:teuthology.orchestra.run.smithi039.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 640 kB/s | 309 kB 00:00 2024-03-16T09:53:37.919 INFO:teuthology.orchestra.run.smithi039.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-03-16T09:53:37.994 INFO:teuthology.orchestra.run.smithi039.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 655 kB/s | 49 kB 00:00 2024-03-16T09:53:38.078 INFO:teuthology.orchestra.run.smithi039.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 815 kB/s | 67 kB 00:00 2024-03-16T09:53:38.187 INFO:teuthology.orchestra.run.smithi039.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-16T09:53:38.220 INFO:teuthology.orchestra.run.smithi039.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-16T09:53:38.296 INFO:teuthology.orchestra.run.smithi039.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-03-16T09:53:38.371 INFO:teuthology.orchestra.run.smithi039.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 586 kB/s | 43 kB 00:00 2024-03-16T09:53:38.404 INFO:teuthology.orchestra.run.smithi039.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-03-16T09:53:38.447 INFO:teuthology.orchestra.run.smithi039.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-03-16T09:53:38.472 INFO:teuthology.orchestra.run.smithi039.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-03-16T09:53:38.522 INFO:teuthology.orchestra.run.smithi039.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-16T09:53:38.564 INFO:teuthology.orchestra.run.smithi039.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-16T09:53:38.606 INFO:teuthology.orchestra.run.smithi039.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-03-16T09:53:38.641 INFO:teuthology.orchestra.run.smithi039.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-03-16T09:53:38.681 INFO:teuthology.orchestra.run.smithi039.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.7 MB/s | 203 kB 00:00 2024-03-16T09:53:38.757 INFO:teuthology.orchestra.run.smithi039.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-16T09:53:38.866 INFO:teuthology.orchestra.run.smithi039.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.1 MB/s | 2.0 MB 00:00 2024-03-16T09:53:38.891 INFO:teuthology.orchestra.run.smithi039.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 81 kB/s | 11 kB 00:00 2024-03-16T09:53:38.933 INFO:teuthology.orchestra.run.smithi039.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 268 kB/s | 18 kB 00:00 2024-03-16T09:53:38.958 INFO:teuthology.orchestra.run.smithi039.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 350 kB/s | 23 kB 00:00 2024-03-16T09:53:39.001 INFO:teuthology.orchestra.run.smithi039.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-03-16T09:53:39.017 INFO:teuthology.orchestra.run.smithi039.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-03-16T09:53:39.093 INFO:teuthology.orchestra.run.smithi039.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 621 kB/s | 46 kB 00:00 2024-03-16T09:53:39.143 INFO:teuthology.orchestra.run.smithi039.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-03-16T09:53:39.168 INFO:teuthology.orchestra.run.smithi039.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-16T09:53:39.219 INFO:teuthology.orchestra.run.smithi039.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 777 kB/s | 58 kB 00:00 2024-03-16T09:53:39.254 INFO:teuthology.orchestra.run.smithi039.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-16T09:53:39.294 INFO:teuthology.orchestra.run.smithi039.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-03-16T09:53:39.319 INFO:teuthology.orchestra.run.smithi039.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-16T09:53:39.370 INFO:teuthology.orchestra.run.smithi039.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-03-16T09:53:39.420 INFO:teuthology.orchestra.run.smithi039.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-03-16T09:53:39.454 INFO:teuthology.orchestra.run.smithi039.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-16T09:53:39.496 INFO:teuthology.orchestra.run.smithi039.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 796 kB/s | 59 kB 00:00 2024-03-16T09:53:39.529 INFO:teuthology.orchestra.run.smithi039.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-03-16T09:53:39.571 INFO:teuthology.orchestra.run.smithi039.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 682 kB/s | 51 kB 00:00 2024-03-16T09:53:39.605 INFO:teuthology.orchestra.run.smithi039.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-16T09:53:39.647 INFO:teuthology.orchestra.run.smithi039.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-16T09:53:39.689 INFO:teuthology.orchestra.run.smithi039.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-16T09:53:39.723 INFO:teuthology.orchestra.run.smithi039.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-03-16T09:53:39.765 INFO:teuthology.orchestra.run.smithi039.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-16T09:53:39.849 INFO:teuthology.orchestra.run.smithi039.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-16T09:53:40.165 INFO:teuthology.orchestra.run.smithi039.stdout:(150/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 10 MB/s | 3.1 MB 00:00 2024-03-16T09:53:40.207 INFO:teuthology.orchestra.run.smithi039.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-16T09:53:40.299 INFO:teuthology.orchestra.run.smithi039.stdout:(152/174): openssl-devel-3.0.7-27.el9.x86_64.rp 648 kB/s | 4.1 MB 00:06 2024-03-16T09:53:40.525 INFO:teuthology.orchestra.run.smithi039.stdout:(153/174): librados2-19.0.0-2162.g3d78f5ba.el9. 4.5 MB/s | 3.4 MB 00:00 2024-03-16T09:53:40.650 INFO:teuthology.orchestra.run.smithi039.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 341 kB/s | 42 kB 00:00 2024-03-16T09:53:40.717 INFO:teuthology.orchestra.run.smithi039.stdout:(155/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.5 MB/s | 2.3 MB 00:00 2024-03-16T09:53:40.768 INFO:teuthology.orchestra.run.smithi039.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.9 MB/s | 1.8 MB 00:00 2024-03-16T09:53:40.793 INFO:teuthology.orchestra.run.smithi039.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 959 kB/s | 71 kB 00:00 2024-03-16T09:53:40.818 INFO:teuthology.orchestra.run.smithi039.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 739 kB/s | 36 kB 00:00 2024-03-16T09:53:40.852 INFO:teuthology.orchestra.run.smithi039.stdout:(159/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.2 MB/s | 248 kB 00:00 2024-03-16T09:53:40.885 INFO:teuthology.orchestra.run.smithi039.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.1 MB/s | 192 kB 00:00 2024-03-16T09:53:40.910 INFO:teuthology.orchestra.run.smithi039.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 399 kB/s | 23 kB 00:00 2024-03-16T09:53:40.944 INFO:teuthology.orchestra.run.smithi039.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-16T09:53:40.969 INFO:teuthology.orchestra.run.smithi039.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-03-16T09:53:41.003 INFO:teuthology.orchestra.run.smithi039.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 155 kB 00:00 2024-03-16T09:53:41.020 INFO:teuthology.orchestra.run.smithi039.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-16T09:53:41.045 INFO:teuthology.orchestra.run.smithi039.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 856 kB/s | 64 kB 00:00 2024-03-16T09:53:41.085 INFO:teuthology.orchestra.run.smithi039.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 449 kB/s | 29 kB 00:00 2024-03-16T09:53:41.185 INFO:teuthology.orchestra.run.smithi039.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 16 MB/s | 2.2 MB 00:00 2024-03-16T09:53:41.235 INFO:teuthology.orchestra.run.smithi039.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 1.2 MB 00:00 2024-03-16T09:53:41.277 INFO:teuthology.orchestra.run.smithi039.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.0 MB/s | 465 kB 00:00 2024-03-16T09:53:41.303 INFO:teuthology.orchestra.run.smithi039.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 468 kB/s | 31 kB 00:00 2024-03-16T09:53:41.378 INFO:teuthology.orchestra.run.smithi039.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.3 MB 00:00 2024-03-16T09:53:41.420 INFO:teuthology.orchestra.run.smithi039.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.6 MB/s | 1.1 MB 00:00 2024-03-16T09:53:42.045 INFO:teuthology.orchestra.run.smithi039.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 100 kB/s | 74 kB 00:00 2024-03-16T09:53:42.058 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:53:42.058 INFO:teuthology.orchestra.run.smithi039.stdout:Total 10 MB/s | 224 MB 00:21 2024-03-16T09:53:42.852 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-16T09:53:43.049 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:53:43.207 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:53:43.237 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:53:43.248 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:53:43.288 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:53:43.341 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:53:43.374 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:53:43.400 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:53:43.442 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:53:43.552 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-03-16T09:53:43.772 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:53:43.850 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:53:43.950 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-03-16T09:53:43.951 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-03-16T09:53:43.979 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:53:44.002 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:53:44.076 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:53:44.106 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:53:44.145 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:53:44.327 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:53:44.386 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:53:44.415 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:53:44.509 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:53:44.532 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:53:44.596 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:53:44.628 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:53:44.650 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:53:44.666 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:53:44.817 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:53:44.848 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:53:44.880 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:53:44.985 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:53:45.083 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:53:45.143 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:53:45.204 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:45.269 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:45.488 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:45.595 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:53:45.880 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:53:45.924 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:53:45.950 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:53:45.986 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:53:46.060 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:53:46.114 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:53:46.152 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:53:46.217 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:53:46.251 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:53:46.295 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:53:46.328 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:53:46.356 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:53:46.424 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:53:46.460 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:53:46.504 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:53:46.532 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:53:46.602 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-03-16T09:53:46.603 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-03-16T09:53:46.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:53:46.655 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:53:46.730 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:53:46.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:53:46.925 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:53:47.435 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:53:47.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:53:47.604 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:53:47.648 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:53:47.656 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:53:47.688 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:53:47.728 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:53:47.767 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:53:47.796 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:53:48.048 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:53:48.205 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:53:48.257 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:53:48.302 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:53:48.457 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:53:48.496 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:53:48.551 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:53:48.624 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:53:48.655 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:53:48.691 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:53:48.738 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:53:48.774 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:53:48.817 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:53:48.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:53:48.992 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:53:49.033 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:53:49.063 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:53:49.104 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:53:49.206 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:53:49.255 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:53:49.295 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:53:49.320 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:53:49.345 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:53:49.417 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:53:49.546 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:53:49.654 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:53:49.695 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:53:49.743 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:53:49.799 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:53:49.984 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:53:50.032 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:53:50.170 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:53:50.214 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:53:50.245 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:53:50.277 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:53:50.305 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:53:50.339 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:53:50.378 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:53:50.409 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:53:50.440 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:53:50.516 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:53:50.550 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:53:50.583 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:53:50.625 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:53:50.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:53:50.749 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:53:50.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:53:50.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:53:50.939 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:53:50.986 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:53:51.093 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:53:51.140 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:53:51.858 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:53:51.912 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:53:51.946 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:53:51.995 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:53:52.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:53:52.130 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:53:52.307 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:53:52.344 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:53:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:53:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:53:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:53:54.096 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-03-16T09:53:54.212 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:53:54.260 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:53:54.379 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:53:54.381 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:53:54.425 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:53:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:53:54.476 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:53:54.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:53:54.522 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:53:54.535 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:53:54.620 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:53:54.674 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:53:54.714 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:53:54.815 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:53:54.849 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:53:54.882 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:53:54.972 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:53:55.003 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:53:55.039 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:53:55.233 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:53:55.270 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:53:55.412 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:53:55.438 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:53:55.499 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:53:55.526 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:53:55.581 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:53:55.593 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:53:55.603 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:53:55.631 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:53:55.706 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:53:56.026 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:53:56.090 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:53:56.132 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:53:56.300 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:53:56.325 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:53:56.425 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:53:56.470 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:53:56.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:53:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:53:56.928 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:53:56.977 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:53:57.029 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:53:57.358 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:53:57.578 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:53:57.667 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:53:57.807 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:53:57.850 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:53:57.897 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:57.955 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:58.294 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:53:58.387 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:53:58.444 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:53:58.482 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:53:58.525 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:53:58.525 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:53:58.526 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:53:58.526 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:53:58.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:53:58.607 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:53:58.644 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:53:58.682 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:53:58.750 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:53:58.869 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:53:58.938 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:53:59.022 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:53:59.131 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:53:59.184 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:53:59.250 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:53:59.299 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:53:59.338 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:53:59.460 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:53:59.499 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:53:59.559 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:53:59.623 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:53:59.755 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:53:59.813 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:53:59.928 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:54:00.114 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:54:00.203 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:54:00.901 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:54:01.003 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:54:01.102 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:54:01.151 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:54:01.176 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:54:01.234 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:54:01.351 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:54:01.416 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:54:01.462 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:54:01.842 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:54:02.036 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:54:02.122 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:54:02.195 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:54:02.398 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:54:02.498 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:54:02.586 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:54:02.687 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:54:02.726 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:54:02.766 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:54:02.836 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:54:02.882 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:54:02.946 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:54:03.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:54:03.220 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:54:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:54:03.320 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:54:03.382 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:54:03.523 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:54:03.566 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:54:03.629 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:54:03.666 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:54:03.699 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:54:03.880 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:54:04.046 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:54:04.179 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:54:04.233 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:54:04.294 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:54:04.361 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:54:04.593 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:54:04.647 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:54:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:54:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:54:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:54:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:54:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:54:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:54:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:54:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:54:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:54:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:54:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:54:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:54:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:54:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:54:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:54:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:54:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:54:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:54:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:54:06.148 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:54:06.208 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:54:07.212 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:54:07.273 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:54:07.307 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:54:07.365 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:54:07.490 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:54:07.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:54:07.790 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:54:07.820 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:54:07.922 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:54:08.758 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:54:08.888 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:54:10.069 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:54:10.069 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-16T09:54:10.069 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-16T09:54:10.069 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-16T09:54:10.070 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-16T09:54:10.070 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-16T09:54:10.070 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-16T09:54:10.070 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-16T09:54:10.070 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:54:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:54:10.249 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:10.249 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:54:10.249 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:10.545 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:54:10.650 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:54:10.688 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:54:10.718 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:54:10.759 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:54:11.153 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:54:11.181 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:54:11.302 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:54:11.337 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:54:11.392 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:54:11.742 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:54:11.767 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:54:11.767 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:11.767 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:54:11.768 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:54:11.768 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:54:11.768 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:11.941 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:54:11.988 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:54:12.010 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:54:12.011 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:12.103 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:54:12.129 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:54:12.447 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:54:12.469 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:12.565 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:54:12.959 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:54:12.980 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:54:13.013 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:54:13.072 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:54:13.095 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:54:13.326 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:54:13.356 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:54:14.314 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:54:14.349 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:54:14.447 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:54:14.602 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:54:14.643 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:54:14.665 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:54:14.665 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:14.665 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:54:14.665 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:54:14.665 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:54:14.666 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:14.724 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:54:14.735 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:54:14.778 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:54:14.782 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:54:14.824 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:54:15.492 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:54:15.537 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:54:15.653 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:54:15.686 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:54:15.709 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:54:15.709 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:15.709 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:54:15.710 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:54:15.710 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:54:15.710 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:15.715 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:54:15.715 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:54:15.715 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:54:15.715 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:15.754 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:54:15.754 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:54:15.776 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:54:15.776 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:15.776 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:54:15.776 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:15.821 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:54:15.844 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:16.164 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:54:16.186 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:54:16.186 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:16.186 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:54:16.186 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:54:16.187 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:54:16.187 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:20.610 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:54:20.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:54:20.681 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:54:20.708 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:54:20.887 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:54:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:54:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:54:21.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:54:21.140 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:54:21.192 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:54:21.216 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:54:21.240 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:54:21.270 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:54:21.316 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:54:21.360 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:54:21.385 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:54:21.996 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:22.041 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:22.061 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:22.867 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:54:22.961 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:54:23.002 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:54:23.082 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:54:23.115 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:54:23.140 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:54:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:54:23.192 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:54:23.192 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:54:23.272 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:54:23.301 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:54:23.301 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:54:23.369 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:54:23.395 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:54:23.429 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:54:23.454 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:54:23.479 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:54:23.512 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:54:23.551 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:54:23.584 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:54:23.609 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:54:23.634 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:54:23.660 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:54:23.685 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:54:23.692 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:23.692 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:23.871 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:23.871 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:54:23.954 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:54:23.979 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:54:24.004 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:54:24.037 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:54:24.063 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:54:24.127 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:54:29.003 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:54:29.004 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:54:29.005 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:54:29.006 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:54:29.007 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:54:29.008 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:54:29.009 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:54:29.010 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:54:29.011 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:54:29.012 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:54:29.013 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:54:29.014 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:54:29.015 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:54:29.016 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:54:29.017 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:54:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:54:29.019 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:54:29.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:54:29.021 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:54:29.021 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:54:29.021 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:54:29.423 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:54:29.423 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-03-16T09:54:29.423 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-03-16T09:54:29.423 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-03-16T09:54:29.423 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-03-16T09:54:29.424 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-03-16T09:54:29.424 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-03-16T09:54:29.424 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-03-16T09:54:29.424 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:29.658 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:54:29.707 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:54:29.707 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:29.708 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:54:29.708 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:54:31.468 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:31.737 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:54:31.806 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:54:31.806 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:31.806 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:54:31.807 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:54:31.807 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:54:31.807 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:31.885 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:54:31.885 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:31.885 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-16T09:54:31.885 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:31.885 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.886 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:54:31.887 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.888 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:54:31.889 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:54:31.890 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:54:31.891 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:54:31.892 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:54:31.893 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:54:31.894 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:54:31.895 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.896 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-16T09:54:31.897 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-16T09:54:32.313 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:54:32.342 DEBUG:teuthology.parallel:result is None 2024-03-16T09:54:32.383 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:54:32.383 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:32.384 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:54:32.384 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:54:32.384 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:54:32.384 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:32.507 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:54:33.132 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:54:33.166 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:54:33.218 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:54:33.321 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:54:33.378 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:54:33.417 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 208 kB/s | 19 MB 01:32 2024-03-16T09:54:33.720 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:54:33.754 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:54:34.902 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:54:34.929 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:54:35.080 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:54:35.259 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:54:35.289 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:54:35.362 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:54:35.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:35.415 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:54:35.454 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:54:35.540 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:54:36.550 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:54:36.584 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:54:36.656 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:54:36.657 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:36.657 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:54:36.657 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:54:36.657 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:54:36.657 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:36.710 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:54:36.782 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:54:36.782 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:36.782 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:54:36.782 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:36.860 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:54:36.936 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:37.311 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:54:37.387 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:37.451 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-16T09:54:38.986 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2024-03-16T09:54:39.979 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-16T09:54:42.434 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:54:42.477 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:54:42.514 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:54:42.553 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:54:42.866 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:54:42.990 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:54:43.043 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:54:43.107 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:54:43.191 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:54:43.247 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:54:43.290 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:54:43.333 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:54:43.386 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:54:43.471 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:54:43.591 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:54:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:54:44.533 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:44.580 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:44.643 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:54:44.878 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-16T09:54:46.089 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:54:46.178 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:54:46.236 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:54:46.367 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:54:46.423 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:54:46.456 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:54:46.482 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:54:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:54:46.524 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:54:46.702 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:54:46.744 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:54:46.744 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:54:46.882 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:54:46.926 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:54:46.980 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:54:47.013 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:54:47.046 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:54:47.125 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:54:47.219 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:54:47.274 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:54:47.309 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:54:47.342 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:54:47.343 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:54:47.344 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:54:47.359 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:54:47.360 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:54:47.361 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:54:47.363 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:54:47.365 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:54:47.368 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:54:47.369 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:54:47.376 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:54:47.409 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:54:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:47.446 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:47.450 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-03-16T09:54:47.470 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:54:47.470 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:54:47.471 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:54:47.472 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:54:47.473 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:54:47.474 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:54:47.475 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:54:47.476 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:54:47.477 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:54:47.478 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:54:47.479 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:54:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:54:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:54:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Install 150 Packages 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 24 Packages 2024-03-16T09:54:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:54:47.485 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 224 M 2024-03-16T09:54:47.485 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-03-16T09:54:47.793 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:54:47.793 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:54:47.915 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:54:47.946 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:54:47.979 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:54:48.036 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:54:48.072 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:54:48.196 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:54:48.429 INFO:teuthology.orchestra.run.smithi194.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-03-16T09:54:48.456 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:54:48.456 INFO:teuthology.orchestra.run.smithi039.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-16T09:54:48.456 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:48.780 INFO:teuthology.orchestra.run.smithi194.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 2.5 MB/s | 890 kB 00:00 2024-03-16T09:54:48.880 INFO:teuthology.orchestra.run.smithi194.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 1.4 MB/s | 145 kB 00:00 2024-03-16T09:54:49.339 INFO:teuthology.orchestra.run.smithi194.stdout:(4/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-03-16T09:54:49.456 INFO:teuthology.orchestra.run.smithi194.stdout:(5/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 4.3 MB/s | 5.4 MB 00:01 2024-03-16T09:54:49.614 INFO:teuthology.orchestra.run.smithi194.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-03-16T09:54:49.964 INFO:teuthology.orchestra.run.smithi194.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 9.2 MB/s | 4.7 MB 00:00 2024-03-16T09:54:50.923 INFO:teuthology.orchestra.run.smithi194.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 10 MB/s | 9.8 MB 00:00 2024-03-16T09:54:50.998 INFO:teuthology.orchestra.run.smithi194.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 337 kB/s | 25 kB 00:00 2024-03-16T09:54:51.448 INFO:teuthology.orchestra.run.smithi194.stdout:(10/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 9.2 MB/s | 17 MB 00:01 2024-03-16T09:54:51.532 INFO:teuthology.orchestra.run.smithi194.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 2.6 MB/s | 221 kB 00:00 2024-03-16T09:54:51.607 INFO:teuthology.orchestra.run.smithi194.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 362 kB/s | 27 kB 00:00 2024-03-16T09:54:51.682 INFO:teuthology.orchestra.run.smithi194.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 424 kB/s | 32 kB 00:00 2024-03-16T09:54:51.832 INFO:teuthology.orchestra.run.smithi194.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 4.9 MB/s | 747 kB 00:00 2024-03-16T09:54:51.916 INFO:teuthology.orchestra.run.smithi194.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.9 MB/s | 163 kB 00:00 2024-03-16T09:54:52.000 INFO:teuthology.orchestra.run.smithi194.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.5 MB/s | 127 kB 00:00 2024-03-16T09:54:52.100 INFO:teuthology.orchestra.run.smithi194.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 4.9 MB/s | 501 kB 00:00 2024-03-16T09:54:52.417 INFO:teuthology.orchestra.run.smithi194.stdout:(18/174): ceph-common-19.0.0-2162.g3d78f5ba.el9 4.9 MB/s | 21 MB 00:04 2024-03-16T09:54:52.525 INFO:teuthology.orchestra.run.smithi194.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 417 kB/s | 45 kB 00:00 2024-03-16T09:54:52.626 INFO:teuthology.orchestra.run.smithi194.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.3 MB/s | 131 kB 00:00 2024-03-16T09:54:52.751 INFO:teuthology.orchestra.run.smithi194.stdout:(21/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 7.9 MB/s | 5.1 MB 00:00 2024-03-16T09:54:52.776 INFO:teuthology.orchestra.run.smithi194.stdout:(22/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.1 MB/s | 162 kB 00:00 2024-03-16T09:54:52.835 INFO:teuthology.orchestra.run.smithi194.stdout:(23/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.8 MB/s | 320 kB 00:00 2024-03-16T09:54:52.868 INFO:teuthology.orchestra.run.smithi194.stdout:(24/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.2 MB/s | 298 kB 00:00 2024-03-16T09:54:52.910 INFO:teuthology.orchestra.run.smithi194.stdout:(25/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.3 MB/s | 99 kB 00:00 2024-03-16T09:54:52.944 INFO:teuthology.orchestra.run.smithi194.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-16T09:54:53.027 INFO:teuthology.orchestra.run.smithi194.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-16T09:54:53.103 INFO:teuthology.orchestra.run.smithi194.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 452 kB/s | 34 kB 00:00 2024-03-16T09:54:53.186 INFO:teuthology.orchestra.run.smithi194.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.6 MB/s | 139 kB 00:00 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:54:53.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:54:53.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:54:53.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:54:53.244 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:54:53.245 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:54:53.246 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:54:53.247 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:54:53.248 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:54:53.249 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:54:53.250 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:54:53.251 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:54:53.252 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:54:53.253 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:54:53.254 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:54:53.255 INFO:teuthology.orchestra.run.smithi194.stdout:(30/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 9.2 MB/s | 3.1 MB 00:00 2024-03-16T09:54:53.495 INFO:teuthology.orchestra.run.smithi194.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 5.8 MB/s | 1.8 MB 00:00 2024-03-16T09:54:53.595 INFO:teuthology.orchestra.run.smithi194.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.4 MB/s | 248 kB 00:00 2024-03-16T09:54:53.687 INFO:teuthology.orchestra.run.smithi194.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 538 kB/s | 49 kB 00:00 2024-03-16T09:54:53.771 INFO:teuthology.orchestra.run.smithi194.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 196 kB/s | 16 kB 00:00 2024-03-16T09:54:53.871 INFO:teuthology.orchestra.run.smithi194.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 2.6 MB/s | 264 kB 00:00 2024-03-16T09:54:54.022 INFO:teuthology.orchestra.run.smithi194.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 9.5 MB/s | 7.4 MB 00:00 2024-03-16T09:54:54.055 INFO:teuthology.orchestra.run.smithi194.stdout:(37/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 4.0 MB/s | 748 kB 00:00 2024-03-16T09:54:54.081 INFO:teuthology.orchestra.run.smithi194.stdout:(38/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 309 kB/s | 18 kB 00:00 2024-03-16T09:54:54.232 INFO:teuthology.orchestra.run.smithi194.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-16T09:54:54.265 INFO:teuthology.orchestra.run.smithi194.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.6 MB/s | 920 kB 00:00 2024-03-16T09:54:54.290 INFO:teuthology.orchestra.run.smithi194.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-16T09:54:54.325 INFO:teuthology.orchestra.run.smithi194.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-16T09:54:54.350 INFO:teuthology.orchestra.run.smithi194.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 537 kB/s | 32 kB 00:00 2024-03-16T09:54:54.751 INFO:teuthology.orchestra.run.smithi194.stdout:(44/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 13 MB/s | 48 MB 00:03 2024-03-16T09:54:54.817 INFO:teuthology.orchestra.run.smithi194.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-03-16T09:54:54.842 INFO:teuthology.orchestra.run.smithi194.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 17 kB 00:00 2024-03-16T09:54:54.868 INFO:teuthology.orchestra.run.smithi194.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 88 kB/s | 10 kB 00:00 2024-03-16T09:54:54.893 INFO:teuthology.orchestra.run.smithi194.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-16T09:54:54.918 INFO:teuthology.orchestra.run.smithi194.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-16T09:54:54.943 INFO:teuthology.orchestra.run.smithi194.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-16T09:54:54.978 INFO:teuthology.orchestra.run.smithi194.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.9 MB/s | 578 kB 00:00 2024-03-16T09:54:55.002 INFO:teuthology.orchestra.run.smithi194.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 46 kB 00:00 2024-03-16T09:54:55.027 INFO:teuthology.orchestra.run.smithi194.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-16T09:54:55.053 INFO:teuthology.orchestra.run.smithi194.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-16T09:54:55.069 INFO:teuthology.orchestra.run.smithi194.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 830 kB/s | 34 kB 00:00 2024-03-16T09:54:55.095 INFO:teuthology.orchestra.run.smithi194.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-16T09:54:55.245 INFO:teuthology.orchestra.run.smithi194.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-16T09:54:55.287 INFO:teuthology.orchestra.run.smithi194.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 7.0 MB/s | 1.5 MB 00:00 2024-03-16T09:54:55.312 INFO:teuthology.orchestra.run.smithi194.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 811 kB 00:00 2024-03-16T09:54:55.337 INFO:teuthology.orchestra.run.smithi194.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-16T09:54:55.379 INFO:teuthology.orchestra.run.smithi194.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-16T09:54:55.547 INFO:teuthology.orchestra.run.smithi194.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-03-16T09:54:55.572 INFO:teuthology.orchestra.run.smithi194.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 334 kB/s | 78 kB 00:00 2024-03-16T09:54:55.656 INFO:teuthology.orchestra.run.smithi194.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-16T09:54:55.681 INFO:teuthology.orchestra.run.smithi194.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-16T09:54:55.706 INFO:teuthology.orchestra.run.smithi194.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-16T09:54:55.731 INFO:teuthology.orchestra.run.smithi194.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-16T09:54:55.756 INFO:teuthology.orchestra.run.smithi194.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-16T09:54:55.823 INFO:teuthology.orchestra.run.smithi194.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 4.8 MB 00:00 2024-03-16T09:54:55.849 INFO:teuthology.orchestra.run.smithi194.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 217 kB 00:00 2024-03-16T09:54:55.874 INFO:teuthology.orchestra.run.smithi194.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 174 kB 00:00 2024-03-16T09:54:55.899 INFO:teuthology.orchestra.run.smithi194.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:54:55.900 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.901 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:54:55.902 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.903 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:54:55.904 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:54:55.905 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:54:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:54:55.907 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:54:55.908 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:54:55.909 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:54:55.910 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:54:55.911 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:54:55.912 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:54:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:54:55.913 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:54:55.913 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-03-16T09:54:55.933 INFO:teuthology.orchestra.run.smithi194.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 702 kB/s | 41 kB 00:00 2024-03-16T09:54:55.958 INFO:teuthology.orchestra.run.smithi194.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-16T09:54:55.983 INFO:teuthology.orchestra.run.smithi194.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-16T09:54:56.000 INFO:teuthology.orchestra.run.smithi194.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-16T09:54:56.025 INFO:teuthology.orchestra.run.smithi194.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-16T09:54:56.042 INFO:teuthology.orchestra.run.smithi194.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 734 kB/s | 43 kB 00:00 2024-03-16T09:54:56.184 INFO:teuthology.orchestra.run.smithi194.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 277 kB/s | 39 kB 00:00 2024-03-16T09:54:56.201 INFO:teuthology.orchestra.run.smithi194.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 262 kB/s | 41 kB 00:00 2024-03-16T09:54:56.351 INFO:teuthology.orchestra.run.smithi194.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-03-16T09:54:56.376 INFO:teuthology.orchestra.run.smithi194.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-16T09:54:56.393 INFO:teuthology.orchestra.run.smithi194.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 297 kB/s | 56 kB 00:00 2024-03-16T09:54:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-16T09:54:56.427 INFO:teuthology.orchestra.run.smithi194.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-16T09:54:56.444 INFO:teuthology.orchestra.run.smithi194.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-16T09:54:56.469 INFO:teuthology.orchestra.run.smithi194.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-03-16T09:54:56.503 INFO:teuthology.orchestra.run.smithi194.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-16T09:54:56.528 INFO:teuthology.orchestra.run.smithi194.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-16T09:54:56.553 INFO:teuthology.orchestra.run.smithi194.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-16T09:54:56.570 INFO:teuthology.orchestra.run.smithi194.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-03-16T09:54:56.612 DEBUG:teuthology.parallel:result is None 2024-03-16T09:54:56.670 INFO:teuthology.orchestra.run.smithi194.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2024-03-16T09:54:56.712 INFO:teuthology.orchestra.run.smithi194.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2024-03-16T09:54:56.738 INFO:teuthology.orchestra.run.smithi194.stdout:(94/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-16T09:54:56.771 INFO:teuthology.orchestra.run.smithi194.stdout:(95/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-16T09:54:56.838 INFO:teuthology.orchestra.run.smithi194.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-03-16T09:54:56.863 INFO:teuthology.orchestra.run.smithi194.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-03-16T09:54:56.930 INFO:teuthology.orchestra.run.smithi194.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-16T09:54:56.955 INFO:teuthology.orchestra.run.smithi194.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-03-16T09:54:56.981 INFO:teuthology.orchestra.run.smithi194.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-16T09:54:57.006 INFO:teuthology.orchestra.run.smithi194.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 695 kB/s | 35 kB 00:00 2024-03-16T09:54:57.031 INFO:teuthology.orchestra.run.smithi194.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-03-16T09:54:57.056 INFO:teuthology.orchestra.run.smithi194.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-16T09:54:57.123 INFO:teuthology.orchestra.run.smithi194.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 12 MB/s | 4.1 MB 00:00 2024-03-16T09:54:57.407 INFO:teuthology.orchestra.run.smithi194.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-03-16T09:54:57.499 INFO:teuthology.orchestra.run.smithi194.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-16T09:54:57.549 INFO:teuthology.orchestra.run.smithi194.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-03-16T09:54:57.691 INFO:teuthology.orchestra.run.smithi194.stdout:(108/174): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-03-16T09:54:57.750 INFO:teuthology.orchestra.run.smithi194.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-03-16T09:54:57.800 INFO:teuthology.orchestra.run.smithi194.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-03-16T09:54:57.842 INFO:teuthology.orchestra.run.smithi194.stdout:(111/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-03-16T09:54:57.876 INFO:teuthology.orchestra.run.smithi194.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 891 kB/s | 67 kB 00:00 2024-03-16T09:54:57.968 INFO:teuthology.orchestra.run.smithi194.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-16T09:54:57.993 INFO:teuthology.orchestra.run.smithi194.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-16T09:54:58.035 INFO:teuthology.orchestra.run.smithi194.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-16T09:54:58.060 INFO:teuthology.orchestra.run.smithi194.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-03-16T09:54:58.085 INFO:teuthology.orchestra.run.smithi194.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-03-16T09:54:58.110 INFO:teuthology.orchestra.run.smithi194.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-03-16T09:54:58.194 INFO:teuthology.orchestra.run.smithi194.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-03-16T09:54:58.244 INFO:teuthology.orchestra.run.smithi194.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-16T09:54:58.278 INFO:teuthology.orchestra.run.smithi194.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-16T09:54:58.320 INFO:teuthology.orchestra.run.smithi194.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-03-16T09:54:58.346 INFO:teuthology.orchestra.run.smithi194.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-16T09:54:58.421 INFO:teuthology.orchestra.run.smithi194.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-16T09:54:58.471 INFO:teuthology.orchestra.run.smithi194.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-03-16T09:54:58.521 INFO:teuthology.orchestra.run.smithi194.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-03-16T09:54:58.555 INFO:teuthology.orchestra.run.smithi194.stdout:(127/174): protobuf-compiler-3.14.0-13.el9.x86_ 576 kB/s | 863 kB 00:01 2024-03-16T09:54:58.580 INFO:teuthology.orchestra.run.smithi194.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-03-16T09:54:58.605 INFO:teuthology.orchestra.run.smithi194.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-03-16T09:54:58.655 INFO:teuthology.orchestra.run.smithi194.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-03-16T09:54:58.698 INFO:teuthology.orchestra.run.smithi194.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-16T09:54:58.723 INFO:teuthology.orchestra.run.smithi194.stdout:(132/174): python3-jaraco-functools-3.5.0-2.el9 116 kB/s | 19 kB 00:00 2024-03-16T09:54:58.748 INFO:teuthology.orchestra.run.smithi194.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-03-16T09:54:58.773 INFO:teuthology.orchestra.run.smithi194.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-16T09:54:58.807 INFO:teuthology.orchestra.run.smithi194.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-03-16T09:54:58.832 INFO:teuthology.orchestra.run.smithi194.stdout:(136/174): python3-natsort-7.1.1-5.el9.noarch.r 529 kB/s | 58 kB 00:00 2024-03-16T09:54:58.857 INFO:teuthology.orchestra.run.smithi194.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-16T09:54:58.883 INFO:teuthology.orchestra.run.smithi194.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-16T09:54:58.916 INFO:teuthology.orchestra.run.smithi194.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-03-16T09:54:58.941 INFO:teuthology.orchestra.run.smithi194.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-03-16T09:54:58.967 INFO:teuthology.orchestra.run.smithi194.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-16T09:54:58.992 INFO:teuthology.orchestra.run.smithi194.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-03-16T09:54:59.019 INFO:teuthology.orchestra.run.smithi194.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 460 kB/s | 36 kB 00:00 2024-03-16T09:54:59.053 INFO:teuthology.orchestra.run.smithi194.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 593 kB/s | 51 kB 00:00 2024-03-16T09:54:59.078 INFO:teuthology.orchestra.run.smithi194.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.6 MB/s | 230 kB 00:00 2024-03-16T09:54:59.137 INFO:teuthology.orchestra.run.smithi194.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-16T09:54:59.154 INFO:teuthology.orchestra.run.smithi194.stdout:(147/174): python3-websocket-client-1.2.3-2.el9 668 kB/s | 90 kB 00:00 2024-03-16T09:54:59.179 INFO:teuthology.orchestra.run.smithi194.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-16T09:54:59.204 INFO:teuthology.orchestra.run.smithi194.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-16T09:54:59.271 INFO:teuthology.orchestra.run.smithi194.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-16T09:54:59.338 INFO:teuthology.orchestra.run.smithi194.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 8.0 MB/s | 321 kB 00:00 2024-03-16T09:54:59.513 INFO:teuthology.orchestra.run.smithi194.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-16T09:54:59.588 INFO:teuthology.orchestra.run.smithi194.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-16T09:54:59.647 INFO:teuthology.orchestra.run.smithi194.stdout:(154/174): librados2-19.0.0-2162.g3d78f5ba.el9. 7.3 MB/s | 3.4 MB 00:00 2024-03-16T09:54:59.672 INFO:teuthology.orchestra.run.smithi194.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 509 kB/s | 42 kB 00:00 2024-03-16T09:54:59.714 INFO:teuthology.orchestra.run.smithi194.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-16T09:54:59.739 INFO:teuthology.orchestra.run.smithi194.stdout:(157/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.6 MB/s | 248 kB 00:00 2024-03-16T09:54:59.756 INFO:teuthology.orchestra.run.smithi194.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 870 kB/s | 36 kB 00:00 2024-03-16T09:54:59.782 INFO:teuthology.orchestra.run.smithi194.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-16T09:54:59.815 INFO:teuthology.orchestra.run.smithi194.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-16T09:54:59.840 INFO:teuthology.orchestra.run.smithi194.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-03-16T09:54:59.866 INFO:teuthology.orchestra.run.smithi194.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-16T09:54:59.891 INFO:teuthology.orchestra.run.smithi194.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-16T09:54:59.916 INFO:teuthology.orchestra.run.smithi194.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-16T09:54:59.941 INFO:teuthology.orchestra.run.smithi194.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-16T09:54:59.967 INFO:teuthology.orchestra.run.smithi194.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 571 kB/s | 29 kB 00:00 2024-03-16T09:55:00.042 INFO:teuthology.orchestra.run.smithi194.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-03-16T09:55:00.084 INFO:teuthology.orchestra.run.smithi194.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-16T09:55:00.151 INFO:teuthology.orchestra.run.smithi194.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.3 MB 00:00 2024-03-16T09:55:00.184 INFO:teuthology.orchestra.run.smithi194.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 4.5 MB/s | 465 kB 00:00 2024-03-16T09:55:00.209 INFO:teuthology.orchestra.run.smithi194.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-03-16T09:55:00.243 INFO:teuthology.orchestra.run.smithi194.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-16T09:55:00.293 INFO:teuthology.orchestra.run.smithi194.stdout:(173/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-03-16T09:55:00.319 INFO:teuthology.orchestra.run.smithi194.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 683 kB/s | 74 kB 00:00 2024-03-16T09:55:00.323 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:55:00.323 INFO:teuthology.orchestra.run.smithi194.stdout:Total 17 MB/s | 224 MB 00:12 2024-03-16T09:55:00.785 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-03-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-03-16T09:55:01.057 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-03-16T09:55:02.763 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-03-16T09:55:02.763 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-03-16T09:55:09.594 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-03-16T09:55:09.714 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:55:09.748 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:55:09.787 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:55:09.797 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:55:09.836 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:55:09.890 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:55:09.932 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:55:09.958 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:55:10.000 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:55:10.319 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:55:10.384 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:55:10.504 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:55:10.535 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:55:10.626 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:55:10.655 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:55:10.696 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:55:10.862 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:55:10.911 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:55:10.940 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:55:11.042 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:55:11.066 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:55:11.129 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:55:11.169 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:55:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:55:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:55:11.408 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:55:11.439 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:55:11.471 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:55:11.583 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:55:11.681 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:55:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:55:11.809 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:55:11.868 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:55:12.097 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:55:12.198 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:55:12.283 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:55:12.314 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:55:12.340 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:55:12.376 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:55:12.450 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:55:12.504 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:55:12.542 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:55:12.606 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:55:12.641 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:55:12.684 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:55:12.717 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:55:12.745 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:55:12.815 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:55:12.859 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:55:12.902 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:55:12.930 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:55:13.001 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:55:13.053 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:55:13.127 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:55:13.258 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:55:13.313 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:55:13.828 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:55:13.910 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:55:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:55:14.037 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:55:14.045 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:55:14.077 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:55:14.117 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:55:14.155 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:55:14.185 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:55:14.411 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:55:14.608 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:55:14.670 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:55:14.715 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:55:14.841 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:55:14.972 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:55:15.249 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:55:15.321 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:55:15.358 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:55:15.395 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:55:15.441 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:55:15.477 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:55:15.520 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:55:15.563 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:55:15.697 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:55:15.742 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:55:15.772 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:55:15.813 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:55:15.909 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:55:16.097 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:55:16.204 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:55:16.355 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:55:16.379 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:55:16.453 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:55:16.589 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:55:16.695 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:55:16.736 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:55:16.785 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:55:16.840 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:55:17.021 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:55:17.074 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:55:17.211 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:55:17.246 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:55:17.279 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:55:17.310 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:55:17.337 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:55:17.372 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:55:17.411 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:55:17.442 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:55:17.472 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:55:17.549 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:55:17.583 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:55:17.616 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:55:17.657 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:55:17.692 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:55:17.773 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:55:17.863 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:55:17.906 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:55:17.938 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:55:17.985 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:55:18.082 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:55:18.130 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:55:18.849 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:55:18.894 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:55:18.928 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:55:18.976 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:55:19.065 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:55:19.095 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:55:19.263 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:55:19.293 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:55:19.370 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:55:20.006 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:55:20.115 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:55:21.309 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:55:21.392 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:55:21.423 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:55:21.454 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:55:21.482 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:55:21.791 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:55:21.820 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:55:21.910 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:55:21.941 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:55:21.977 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:55:22.435 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:55:22.456 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:55:22.511 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:55:22.533 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:55:24.734 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:55:24.771 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:55:25.306 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:55:25.349 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:55:25.430 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:55:25.430 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:55:25.430 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:55:25.430 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:25.450 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-03-16T09:55:37.589 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:37.761 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:55:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:55:37.776 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:37.776 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:55:37.776 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:39.247 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:55:39.272 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:39.498 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:55:39.520 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:55:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:55:39.979 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:39.979 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:55:39.979 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:55:39.980 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:55:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:40.075 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:55:40.459 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:55:40.490 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:55:40.513 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:55:40.565 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:55:40.588 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:55:40.820 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:55:40.847 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:55:41.808 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:55:41.839 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:55:41.938 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:55:42.095 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:55:42.124 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:55:42.147 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:55:42.147 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:42.148 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:55:42.148 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:55:42.148 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:55:42.148 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:42.193 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:55:42.203 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:55:42.239 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:55:43.100 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:55:43.127 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:55:43.149 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:43.194 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:55:43.215 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:55:43.215 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:43.216 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:55:43.216 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:43.263 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:55:43.286 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:43.597 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:55:43.620 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:55:43.620 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:55:43.620 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:55:43.620 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:55:43.621 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:55:43.621 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:48.051 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:55:48.092 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:55:48.122 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:55:48.179 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:55:48.353 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:55:48.467 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:55:48.507 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:55:48.545 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:55:48.589 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:55:48.632 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:55:48.657 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:55:48.697 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:55:48.727 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:55:48.781 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:55:48.825 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:55:48.850 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:55:49.377 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:55:49.414 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:55:49.435 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:55:50.245 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:55:50.326 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:55:50.366 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:55:50.447 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:55:50.479 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:55:50.504 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:55:50.538 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:55:50.564 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:55:50.564 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:55:50.644 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:55:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:55:50.674 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:55:50.783 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:55:50.809 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:55:50.842 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:55:50.868 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:55:50.893 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:55:50.925 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:55:50.964 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:55:50.997 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:55:51.022 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:55:51.048 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:55:51.073 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:55:51.098 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:55:51.105 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:55:51.105 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:55:51.284 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:55:51.284 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:55:51.367 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:55:51.392 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:55:51.417 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:55:51.450 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:55:51.476 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:55:51.541 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:55:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:55:56.170 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:55:56.171 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:55:56.172 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:55:56.173 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:55:56.174 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:55:56.175 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:55:56.176 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:55:56.177 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:55:56.178 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:55:56.179 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:55:56.180 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:55:56.181 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:55:56.182 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:55:56.183 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:55:56.184 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:55:56.185 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:55:56.186 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:55:56.187 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:55:59.193 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:55:59.194 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.195 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.196 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.197 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:55:59.198 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:55:59.199 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:55:59.200 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:55:59.201 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:55:59.202 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:55:59.203 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:55:59.204 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:55:59.205 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:55:59.205 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:55:59.205 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-03-16T09:55:59.779 DEBUG:teuthology.parallel:result is None 2024-03-16T09:55:59.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:55:59.998 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:56:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.076 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.076 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:56:00.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:56:00.231 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:56:00.257 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.257 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.257 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:56:00.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:56:00.473 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:56:00.494 INFO:teuthology.orchestra.run.smithi194.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.494 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:56:00.494 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:56:00.495 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T09:56:00.496 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:56:00.496 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:56:00.607 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:56:00.607 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:56:00.642 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:56:00.642 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:56:00.673 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T09:56:00.673 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:56:00.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:56:00.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:56:00.922 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:56:00.922 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:56:00.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:56:01.023 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:56:01.023 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:56:01.050 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:56:01.116 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T09:56:01.116 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:56:01.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:56:01.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:56:01.366 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:56:01.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:56:01.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:56:01.468 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:56:01.468 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:56:01.496 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:56:01.562 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T09:56:01.562 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:56:01.563 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:56:01.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:56:01.770 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:56:01.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:56:01.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:56:01.872 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:56:01.872 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:56:01.899 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:56:01.968 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-16T09:56:02.068 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'subvols': {'create': 2}} 2024-03-16T09:56:02.069 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:56:02.069 INFO:tasks.cephadm:Cluster fsid is 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:56:02.069 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-16T09:56:02.070 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.105', 'mon.c': '172.21.15.194'} 2024-03-16T09:56:02.070 INFO:tasks.cephadm:First mon is mon.a on smithi039 2024-03-16T09:56:02.070 INFO:tasks.cephadm:First mgr is x 2024-03-16T09:56:02.070 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-16T09:56:02.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-03-16T09:56:02.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-03-16T09:56:02.207 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-03-16T09:56:02.234 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-16T09:56:02.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:56:02.380 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77658/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2162-g3d78f5ba', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2162.g3d78f5ba'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 01:33:04.210510', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240314.183243', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-16T09:56:02.518 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20240314.183243, sha1 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:56:02.519 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm 2024-03-16T09:56:02.520 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:03.235 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:56 /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:03.235 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:03.882 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:56 /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:03.882 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:04.528 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:56 /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:04.528 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:04.591 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:04.609 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:56:04.647 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af on all hosts... 2024-03-16T09:56:04.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:56:04.654 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:56:04.657 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:56:05.459 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:56:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:56:05.904 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:56:31.948 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:56:31.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:56:31.948 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:56:31.948 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-03-16T09:56:31.948 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:56:31.949 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-03-16T09:56:31.949 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:56:37.784 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-16T09:56:37.785 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-16T09:56:41.547 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T09:56:41.547 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:56:41.548 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:56:41.548 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-03-16T09:56:41.548 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:56:41.548 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T09:56:41.548 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T09:56:41.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-03-16T09:56:41.714 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-03-16T09:56:41.749 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-03-16T09:56:41.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-03-16T09:56:41.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-03-16T09:56:41.919 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-03-16T09:56:41.951 INFO:tasks.cephadm:Writing seed config... 2024-03-16T09:56:41.952 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-16T09:56:41.953 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-16T09:56:41.954 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-16T09:56:41.955 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-16T09:56:41.956 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-16T09:56:41.957 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:56:41.957 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-16T09:56:42.017 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 65a28618-e37b-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-16T09:56:42.018 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service 2024-03-16T09:56:42.066 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x.service 2024-03-16T09:56:42.112 INFO:tasks.cephadm:Bootstrapping... 2024-03-16T09:56:42.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -v bootstrap --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:56:42.394 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:56:42.394 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', '-v', 'bootstrap', '--fsid', '65a28618-e37b-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2024-03-16T09:56:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:56:42.457 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-16T09:56:42.457 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2024-03-16T09:56:42.514 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:56:42.514 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2024-03-16T09:56:42.515 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2024-03-16T09:56:42.530 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T09:56:42.531 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T09:56:42.547 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T09:56:42.548 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:56:42.571 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-03-16T09:56:42.590 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-03-16T09:56:42.591 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-03-16T09:56:42.591 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2024-03-16T09:56:42.645 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:56:42.645 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-16T09:56:42.645 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2024-03-16T09:56:42.645 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2024-03-16T09:56:42.661 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T09:56:42.661 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T09:56:42.678 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T09:56:42.678 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:56:42.700 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-03-16T09:56:42.720 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-03-16T09:56:42.721 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-03-16T09:56:42.721 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2024-03-16T09:56:42.722 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:56:42.722 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 140409833291200 on /run/cephadm/65a28618-e37b-11ee-95c9-87774f69a715.lock 2024-03-16T09:56:42.722 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 140409833291200 acquired on /run/cephadm/65a28618-e37b-11ee-95c9-87774f69a715.lock 2024-03-16T09:56:42.724 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2024-03-16T09:56:42.724 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2024-03-16T09:56:42.725 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-03-16T09:56:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.39 metric 100 2024-03-16T09:56:42.741 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2024-03-16T09:56:42.754 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-16T09:56:42.754 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-16T09:56:42.754 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-16T09:56:42.764 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-16T09:56:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-16T09:56:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T09:56:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-16T09:56:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-03-16T09:56:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T09:56:42.768 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-03-16T09:56:42.768 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-03-16T09:56:42.769 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-16T09:56:42.771 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-16T09:56:42.774 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:56:43.401 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T09:56:43.401 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:56:43.401 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2024-03-16T09:56:43.401 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:66487374cb611c95435850c17b53ebb64c22ed9555f72cd43d678b60262a12b1 2024-03-16T09:56:43.401 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-16T09:56:43.402 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T09:56:43.402 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-16T09:56:44.759 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T09:56:44.759 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T09:56:44.759 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2024-03-16T09:56:54.943 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2024-03-16T09:56:54.944 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2024-03-16T09:56:56.091 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDnbPVlD1YyGRAAwj3TAaJSRJ51HQqW3J4gdw== 2024-03-16T09:56:57.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDobPVl1NpRIBAANLINPVwjsMMBVNzX99+aEw== 2024-03-16T09:56:58.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDpbPVl59mlJRAAzwPGzk/ZIm8wkEIZFJivyg== 2024-03-16T09:56:58.278 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2024-03-16T09:56:59.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T09:56:59.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-16T09:56:59.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:56:59.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:56:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2024-03-16T09:57:00.933 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2024-03-16T09:57:01.269 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-16T09:57:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-16T09:57:01.809 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target. 2024-03-16T09:57:01.810 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target. 2024-03-16T09:57:02.207 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a 2024-03-16T09:57:02.207 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-16T09:57:02.509 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target.wants/ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715@.service. 2024-03-16T09:57:03.169 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:02 smithi039 podman[31120]: 2024-03-16 09:57:02.93075397 +0000 UTC m=+0.144307733 container create e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T09:57:03.170 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 podman[31120]: 2024-03-16 09:57:03.167812682 +0000 UTC m=+0.381366452 container init e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:57:03.445 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:57:03.446 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-03-16T09:57:03.471 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:57:03.471 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:57:03.472 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-03-16T09:57:03.472 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T09:57:03.474 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2024-03-16T09:57:03.474 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2024-03-16T09:57:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 podman[31120]: 2024-03-16 09:57:03.178593983 +0000 UTC m=+0.392147749 container start e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T09:57:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 ceph-mon[31139]: mkfs 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 ceph-mon[31139]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T09:57:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 bash[31120]: e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 2024-03-16T09:57:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:03 smithi039 systemd[1]: Started Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:57:04.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T09:57:04.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: monmap epoch 1 2024-03-16T09:57:04.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:04.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: last_changed 2024-03-16T09:56:58.735570+0000 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: min_mon_release 19 (squid) 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: election_strategy: 1 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: fsmap 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: mgrmap e1: no daemons active 2024-03-16T09:57:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:04 smithi039 ceph-mon[31139]: from='client.? 172.21.15.39:0/608784555' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-16T09:57:05.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.892254s) 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:05.724 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.070+0000 7fbdafe00700 1 Processor -- start 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.070+0000 7fbdafe00700 1 -- start start 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.070+0000 7fbdafe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.070+0000 7fbdafe00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810a740 con 0x7fbda8109d50 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.071+0000 7fbdad400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.071+0000 7fbdad400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48300/0 (socket says 172.21.15.39:48300) 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.072+0000 7fbdad400700 1 -- 172.21.15.39:0/363391477 learned_addr learned my addr 172.21.15.39:0/363391477 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.072+0000 7fbdad400700 1 -- 172.21.15.39:0/363391477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda810af50 con 0x7fbda8109d50 2024-03-16T09:57:05.725 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.073+0000 7fbdad400700 1 --2- 172.21.15.39:0/363391477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf02204ced282216 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.073+0000 7fbda7e00700 1 -- 172.21.15.39:0/363391477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbda8109d50 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.073+0000 7fbda7e00700 1 -- 172.21.15.39:0/363391477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c003a20 con 0x7fbda8109d50 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.073+0000 7fbda7e00700 1 -- 172.21.15.39:0/363391477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd9c003d20 con 0x7fbda8109d50 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 -- 172.21.15.39:0/363391477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 msgr2=0x7fbda810a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 --2- 172.21.15.39:0/363391477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009e80 tx=0x7fbd9c00b330 comp rx=0 tx=0).stop 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 -- 172.21.15.39:0/363391477 shutdown_connections 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 --2- 172.21.15.39:0/363391477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda810a170 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 -- 172.21.15.39:0/363391477 >> 172.21.15.39:0/363391477 conn(0x7fbda807d120 msgr2=0x7fbda807d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 -- 172.21.15.39:0/363391477 shutdown_connections 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.074+0000 7fbdafe00700 1 -- 172.21.15.39:0/363391477 wait complete. 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.075+0000 7fbdafe00700 1 Processor -- start 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.075+0000 7fbdafe00700 1 -- start start 2024-03-16T09:57:05.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.076+0000 7fbdafe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.076+0000 7fbdafe00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810b480 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.076+0000 7fbdad400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.076+0000 7fbdad400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48314/0 (socket says 172.21.15.39:48314) 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.076+0000 7fbdad400700 1 -- 172.21.15.39:0/608784555 learned_addr learned my addr 172.21.15.39:0/608784555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbdad400700 1 -- 172.21.15.39:0/608784555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda819da20 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbdad400700 1 --2- 172.21.15.39:0/608784555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c004000 tx=0x7fbd9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd9c022020 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c018040 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda819dcb0 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.077+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a09a0 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.078+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd9c025020 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.078+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd9c00f050 con 0x7fbda8109d50 2024-03-16T09:57:05.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.079+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda8063900 con 0x7fbda8109d50 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.082+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbd9c02c030 con 0x7fbda8109d50 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.163+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fbda802ddc0 con 0x7fbda8109d50 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.164+0000 7fbda6000700 1 -- 172.21.15.39:0/608784555 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fbd9c01e480 con 0x7fbda8109d50 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.165+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 msgr2=0x7fbda819d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.165+0000 7fbdafe00700 1 --2- 172.21.15.39:0/608784555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c004000 tx=0x7fbd9c004ad0 comp rx=0 tx=0).stop 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.165+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 shutdown_connections 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.165+0000 7fbdafe00700 1 --2- 172.21.15.39:0/608784555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda8109d50 0x7fbda819d4e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.165+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 >> 172.21.15.39:0/608784555 conn(0x7fbda807d120 msgr2=0x7fbda8194140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.166+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 shutdown_connections 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:04.166+0000 7fbdafe00700 1 -- 172.21.15.39:0/608784555 wait complete. 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2024-03-16T09:57:05.728 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2024-03-16T09:57:06.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:06 smithi039 ceph-mon[31139]: from='client.? 172.21.15.39:0/3957111935' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T09:57:06.752 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:06 smithi039 ceph-mon[31139]: from='client.? 172.21.15.39:0/3957111935' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T09:57:07.032 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.318+0000 7f0435200700 1 Processor -- start 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.318+0000 7f0435200700 1 -- start start 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.319+0000 7f0435200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.319+0000 7f0435200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f043010a590 con 0x7f0430109ba0 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.319+0000 7f042f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:07.033 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.320+0000 7f042f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48316/0 (socket says 172.21.15.39:48316) 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.320+0000 7f042f400700 1 -- 172.21.15.39:0/1888488616 learned_addr learned my addr 172.21.15.39:0/1888488616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.320+0000 7f042f400700 1 -- 172.21.15.39:0/1888488616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043010ada0 con 0x7f0430109ba0 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.321+0000 7f042f400700 1 --2- 172.21.15.39:0/1888488616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f0420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f24ead404d710f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.321+0000 7f042e000700 1 -- 172.21.15.39:0/1888488616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0420004160 con 0x7f0430109ba0 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.321+0000 7f042e000700 1 -- 172.21.15.39:0/1888488616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420004300 con 0x7f0430109ba0 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.321+0000 7f042e000700 1 -- 172.21.15.39:0/1888488616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04200036a0 con 0x7f0430109ba0 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 -- 172.21.15.39:0/1888488616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 msgr2=0x7f0430109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 --2- 172.21.15.39:0/1888488616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f04200098a0 tx=0x7f0420009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 -- 172.21.15.39:0/1888488616 shutdown_connections 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 --2- 172.21.15.39:0/1888488616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f0430109fc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 -- 172.21.15.39:0/1888488616 >> 172.21.15.39:0/1888488616 conn(0x7f043007cf80 msgr2=0x7f043007d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 -- 172.21.15.39:0/1888488616 shutdown_connections 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.322+0000 7f0435200700 1 -- 172.21.15.39:0/1888488616 wait complete. 2024-03-16T09:57:07.034 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.323+0000 7f0435200700 1 Processor -- start 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.323+0000 7f0435200700 1 -- start start 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f0435200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f0435200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f043010b2d0 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f042f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f042f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48326/0 (socket says 172.21.15.39:48326) 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f042f400700 1 -- 172.21.15.39:0/3957111935 learned_addr learned my addr 172.21.15.39:0/3957111935 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.324+0000 7f042f400700 1 -- 172.21.15.39:0/3957111935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043019d850 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f042f400700 1 --2- 172.21.15.39:0/3957111935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c00 tx=0x7f0420004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0420003db0 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420011630 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043019dae0 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0420011950 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301a0780 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.325+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0420011af0 con 0x7f0430109ba0 2024-03-16T09:57:07.035 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.326+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f042001c450 con 0x7f0430109ba0 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.326+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430063900 con 0x7f0430109ba0 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.329+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0420032020 con 0x7f0430109ba0 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.406+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f043002dcf0 con 0x7f0430109ba0 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.414+0000 7f0427e00700 1 -- 172.21.15.39:0/3957111935 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f0420004260 con 0x7f0430109ba0 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.415+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 msgr2=0x7f043019d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.415+0000 7f0435200700 1 --2- 172.21.15.39:0/3957111935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0420000c00 tx=0x7f0420004750 comp rx=0 tx=0).stop 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.416+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 shutdown_connections 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.416+0000 7f0435200700 1 --2- 172.21.15.39:0/3957111935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0430109ba0 0x7f043019d310 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.416+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 >> 172.21.15.39:0/3957111935 conn(0x7f043007cf80 msgr2=0x7f0430193ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.416+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 shutdown_connections 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:06.416+0000 7f0435200700 1 -- 172.21.15.39:0/3957111935 wait complete. 2024-03-16T09:57:07.036 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2024-03-16T09:57:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:07 smithi039 ceph-mon[31139]: from='client.? 172.21.15.39:0/257581126' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:57:08.447 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.625+0000 7f0387a00700 1 Processor -- start 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.626+0000 7f0387a00700 1 -- start start 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.626+0000 7f0387a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.626+0000 7f0387a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f038010a7a0 con 0x7f0380109db0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.627+0000 7f0385000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.627+0000 7f0385000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48330/0 (socket says 172.21.15.39:48330) 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.627+0000 7f0385000700 1 -- 172.21.15.39:0/3990179751 learned_addr learned my addr 172.21.15.39:0/3990179751 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.627+0000 7f0385000700 1 -- 172.21.15.39:0/3990179751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038010afb0 con 0x7f0380109db0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.627+0000 7f0385000700 1 --2- 172.21.15.39:0/3990179751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f0374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56bc49736c580e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.628+0000 7f037f400700 1 -- 172.21.15.39:0/3990179751 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0374004160 con 0x7f0380109db0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.628+0000 7f037f400700 1 -- 172.21.15.39:0/3990179751 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0374004300 con 0x7f0380109db0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.628+0000 7f037f400700 1 -- 172.21.15.39:0/3990179751 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0374011b50 con 0x7f0380109db0 2024-03-16T09:57:08.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.628+0000 7f0387a00700 1 -- 172.21.15.39:0/3990179751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 msgr2=0x7f038010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.628+0000 7f0387a00700 1 --2- 172.21.15.39:0/3990179751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f0374009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.629+0000 7f0387a00700 1 -- 172.21.15.39:0/3990179751 shutdown_connections 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.629+0000 7f0387a00700 1 --2- 172.21.15.39:0/3990179751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038010a1d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.629+0000 7f0387a00700 1 -- 172.21.15.39:0/3990179751 >> 172.21.15.39:0/3990179751 conn(0x7f038007d1b0 msgr2=0x7f038007d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.629+0000 7f0387a00700 1 -- 172.21.15.39:0/3990179751 shutdown_connections 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.629+0000 7f0387a00700 1 -- 172.21.15.39:0/3990179751 wait complete. 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.630+0000 7f0387a00700 1 Processor -- start 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.630+0000 7f0387a00700 1 -- start start 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.630+0000 7f0387a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.630+0000 7f0387a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f038010e360 con 0x7f0380109db0 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.630+0000 7f0385000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.631+0000 7f0385000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48342/0 (socket says 172.21.15.39:48342) 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.631+0000 7f0385000700 1 -- 172.21.15.39:0/257581126 learned_addr learned my addr 172.21.15.39:0/257581126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:08.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.631+0000 7f0385000700 1 -- 172.21.15.39:0/257581126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038019e2e0 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.631+0000 7f0385000700 1 --2- 172.21.15.39:0/257581126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0374011850 tx=0x7f0374011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0374003af0 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0374020070 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03740195a0 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038019e570 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03801a1260 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0374019be0 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.632+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0374011e00 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.633+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0380063900 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.636+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0374003c90 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.713+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f03801a1430 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.714+0000 7f037d600700 1 -- 172.21.15.39:0/257581126 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0374025460 con 0x7f0380109db0 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.715+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 msgr2=0x7f038019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.715+0000 7f0387a00700 1 --2- 172.21.15.39:0/257581126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0374011850 tx=0x7f0374011c10 comp rx=0 tx=0).stop 2024-03-16T09:57:08.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.715+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 shutdown_connections 2024-03-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.715+0000 7f0387a00700 1 --2- 172.21.15.39:0/257581126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0380109db0 0x7f038019dda0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.716+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 >> 172.21.15.39:0/257581126 conn(0x7f038007d1b0 msgr2=0x7f03801074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.716+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 shutdown_connections 2024-03-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:07.716+0000 7f0387a00700 1 -- 172.21.15.39:0/257581126 wait complete. 2024-03-16T09:57:08.451 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2024-03-16T09:57:08.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:08 smithi039 systemd[1]: Stopping Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T09:57:09.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a[31135]: 2024-03-16T09:57:08.938+0000 7fb5b1600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:57:09.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a[31135]: 2024-03-16T09:57:08.938+0000 7fb5b1600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-16T09:57:09.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:09 smithi039 podman[31382]: 2024-03-16 09:57:09.011462948 +0000 UTC m=+0.199444335 container died e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-16T09:57:09.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:09 smithi039 podman[31382]: 2024-03-16 09:57:09.424040999 +0000 UTC m=+0.612022372 container cleanup e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T09:57:09.585 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:09 smithi039 bash[31382]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a 2024-03-16T09:57:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:09 smithi039 podman[31397]: 2024-03-16 09:57:09.583527535 +0000 UTC m=+0.546864148 container remove e7c8451db48478c6f55e7c2192fd8875d932b5869deed277f953bb09702e2132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-16T09:57:10.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-16T09:57:10.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 systemd[1]: Stopped Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:57:10.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service: Consumed 1.959s CPU time. 2024-03-16T09:57:10.286 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 systemd[1]: Starting Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T09:57:10.727 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 podman[31491]: 2024-03-16 09:57:10.357335967 +0000 UTC m=+0.033225178 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:57:11.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 podman[31491]: 2024-03-16 09:57:10.726080045 +0000 UTC m=+0.401969266 container create 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307) 2024-03-16T09:57:11.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 podman[31491]: 2024-03-16 09:57:10.955330651 +0000 UTC m=+0.631219953 container init 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T09:57:11.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:10 smithi039 podman[31491]: 2024-03-16 09:57:10.966451706 +0000 UTC m=+0.642340960 container start 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:57:11.055 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-16T09:57:11.315 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T09:57:11.315 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T09:57:11.315 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: pidfile_write: ignore empty --pid-file 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: load: jerasure load: lrc 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: RocksDB version: 7.9.2 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Git sha 0 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: DB SUMMARY 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: DB Session ID: S42U76S1HVWRVLTAIWFA 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: CURRENT file: CURRENT 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104214 ; 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.error_if_exists: 0 2024-03-16T09:57:11.316 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.create_if_missing: 0 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.paranoid_checks: 1 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.env: 0x562e16028c60 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.info_log: 0x562e18819b00 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.statistics: (nil) 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.use_fsync: 0 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_log_file_size: 0 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T09:57:11.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_fallocate: 1 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.use_direct_reads: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.db_log_dir: 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.wal_dir: 2024-03-16T09:57:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.write_buffer_manager: 0x562e187ec1e0 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T09:57:11.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.unordered_write: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.row_cache: None 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.wal_filter: None 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.two_write_queues: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T09:57:11.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.wal_compression: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.atomic_flush: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.log_readahead_size: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_background_jobs: 2 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_background_compactions: -1 2024-03-16T09:57:11.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_subcompactions: 1 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_open_files: -1 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_background_flushes: -1 2024-03-16T09:57:11.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Compression algorithms supported: 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kZSTD supported: 0 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kXpressCompression supported: 0 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kZlibCompression supported: 1 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kSnappyCompression supported: 1 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kLZ4Compression supported: 1 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: kBZip2Compression supported: 0 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T09:57:11.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.merge_operator: 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_filter: None 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e18819c20) 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2024-03-16T09:57:11.324 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T09:57:11.325 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x562e1881f090 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2024-03-16T09:57:11.326 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T09:57:11.327 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression: NoCompression 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.num_levels: 7 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T09:57:11.328 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T09:57:11.329 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T09:57:11.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.table_properties_collectors: 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.inplace_update_support: 0 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T09:57:11.331 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.bloom_locality: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.max_successive_merges: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.ttl: 2592000 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enable_blob_files: false 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.min_blob_size: 0 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T09:57:11.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e2989eba-e91f-45a4-9f08-8dbe92f8d390 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583031026571, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583031028368, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97383, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13945, "raw_average_key_size": 49, "raw_value_size": 89754, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710583031, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e2989eba-e91f-45a4-9f08-8dbe92f8d390", "db_session_id": "S42U76S1HVWRVLTAIWFA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583031028562, "job": 1, "event": "recovery_finished"} 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e18912000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: DB pointer 0x562e188fc000 2024-03-16T09:57:11.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T09:57:11.334 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 98.90 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 98.90 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T09:57:11.335 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 8.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 8.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x562e1881f090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???) e1 preinit fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).mds e1 new map 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).mds e1 print_map 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2024-03-16T09:57:11.336 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 bash[31491]: 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 systemd[1]: Started Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:57:11.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: monmap epoch 1 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: last_changed 2024-03-16T09:56:58.735570+0000 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: min_mon_release 19 (squid) 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: election_strategy: 1 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: fsmap 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T09:57:11.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:11 smithi039 ceph-mon[31510]: mgrmap e1: no daemons active 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.609+0000 7ffa0b800700 1 Processor -- start 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.610+0000 7ffa0b800700 1 -- start start 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.610+0000 7ffa0b800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.610+0000 7ffa0b800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04108580 con 0x7ffa04107b90 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.611+0000 7ffa08e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.611+0000 7ffa08e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44222/0 (socket says 172.21.15.39:44222) 2024-03-16T09:57:12.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.611+0000 7ffa08e00700 1 -- 172.21.15.39:0/241925554 learned_addr learned my addr 172.21.15.39:0/241925554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.612+0000 7ffa08e00700 1 -- 172.21.15.39:0/241925554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa04108d90 con 0x7ffa04107b90 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.612+0000 7ffa08e00700 1 --2- 172.21.15.39:0/241925554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009e80 tx=0x7ff9f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d7153c0c0b9669 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.612+0000 7ffa03400700 1 -- 172.21.15.39:0/241925554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9f8017070 con 0x7ffa04107b90 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.612+0000 7ffa03400700 1 -- 172.21.15.39:0/241925554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9f800b8c0 con 0x7ffa04107b90 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.612+0000 7ffa03400700 1 -- 172.21.15.39:0/241925554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9f80045c0 con 0x7ffa04107b90 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.613+0000 7ffa0b800700 1 -- 172.21.15.39:0/241925554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 msgr2=0x7ffa04107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.613+0000 7ffa0b800700 1 --2- 172.21.15.39:0/241925554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009e80 tx=0x7ff9f800b330 comp rx=0 tx=0).stop 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.613+0000 7ffa0b800700 1 -- 172.21.15.39:0/241925554 shutdown_connections 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.613+0000 7ffa0b800700 1 --2- 172.21.15.39:0/241925554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa04107fb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.613+0000 7ffa0b800700 1 -- 172.21.15.39:0/241925554 >> 172.21.15.39:0/241925554 conn(0x7ffa04103150 msgr2=0x7ffa04105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.614+0000 7ffa0b800700 1 -- 172.21.15.39:0/241925554 shutdown_connections 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.614+0000 7ffa0b800700 1 -- 172.21.15.39:0/241925554 wait complete. 2024-03-16T09:57:12.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.615+0000 7ffa0b800700 1 Processor -- start 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.615+0000 7ffa0b800700 1 -- start start 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.615+0000 7ffa0b800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.615+0000 7ffa0b800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0410c140 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.616+0000 7ffa08e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.616+0000 7ffa08e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44232/0 (socket says 172.21.15.39:44232) 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.616+0000 7ffa08e00700 1 -- 172.21.15.39:0/4010179060 learned_addr learned my addr 172.21.15.39:0/4010179060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.616+0000 7ffa08e00700 1 -- 172.21.15.39:0/4010179060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0419e260 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa08e00700 1 --2- 172.21.15.39:0/4010179060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80072a0 tx=0x7ff9f800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9f8020020 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0419e4f0 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9f8003b50 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9f8017040 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.617+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa041a11d0 con 0x7ffa04107b90 2024-03-16T09:57:12.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.618+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff9f8023020 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.618+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9f800f050 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.618+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa04063900 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.622+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff9f802a020 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.700+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ffa041a1800 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.704+0000 7ffa01600700 1 -- 172.21.15.39:0/4010179060 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff9f8003d00 con 0x7ffa04107b90 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 msgr2=0x7ffa0419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 --2- 172.21.15.39:0/4010179060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80072a0 tx=0x7ff9f800bd50 comp rx=0 tx=0).stop 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 shutdown_connections 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 --2- 172.21.15.39:0/4010179060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa04107b90 0x7ffa0419dd20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 >> 172.21.15.39:0/4010179060 conn(0x7ffa04103150 msgr2=0x7ffa04103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 shutdown_connections 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:11.706+0000 7ffa0b800700 1 -- 172.21.15.39:0/4010179060 wait complete. 2024-03-16T09:57:12.362 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-16T09:57:12.376 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:57:12.377 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2024-03-16T09:57:12.378 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-16T09:57:12.379 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-16T09:57:12.783 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x 2024-03-16T09:57:12.783 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-16T09:57:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:12 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4010179060' entity='client.admin' 2024-03-16T09:57:13.057 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715.target.wants/ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-65a28618-e37b-11ee-95c9-87774f69a715@.service. 2024-03-16T09:57:13.443 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 systemd[1]: Starting Ceph mgr.x for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T09:57:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 podman[31734]: 2024-03-16 09:57:13.346615653 +0000 UTC m=+0.031727319 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:57:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 podman[31734]: 2024-03-16 09:57:13.464182515 +0000 UTC m=+0.149294196 container create d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-16T09:57:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 podman[31734]: 2024-03-16 09:57:13.67443025 +0000 UTC m=+0.359541955 container init d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T09:57:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 podman[31734]: 2024-03-16 09:57:13.686083754 +0000 UTC m=+0.371195437 container start d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:57:13.795 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:57:13.795 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-03-16T09:57:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:57:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:57:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-03-16T09:57:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T09:57:13.829 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:57:13.830 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-03-16T09:57:13.846 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:57:13.846 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:57:13.846 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-03-16T09:57:13.847 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-16T09:57:13.847 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2024-03-16T09:57:13.847 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2024-03-16T09:57:14.033 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 bash[31734]: d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f 2024-03-16T09:57:14.033 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:13 smithi039 systemd[1]: Started Ceph mgr.x for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:57:14.546 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:14.494+0000 7f590acca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:57:14.881 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:14 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/328904865' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:57:14.881 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:14.625+0000 7f590acca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:57:15.254 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:57:15.257 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:57:15.258 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:57:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:57:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:57:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:57:15.267 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:15.268 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:57:03.275227+0000", 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.435+0000 7faaec600700 1 Processor -- start 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.435+0000 7faaec600700 1 -- start start 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.435+0000 7faaec600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.435+0000 7faaec600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faae410af40 con 0x7faae4108610 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.436+0000 7faae9c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.436+0000 7faae9c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44262/0 (socket says 172.21.15.39:44262) 2024-03-16T09:57:15.269 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.436+0000 7faae9c00700 1 -- 172.21.15.39:0/3420396395 learned_addr learned my addr 172.21.15.39:0/3420396395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.437+0000 7faae9c00700 1 -- 172.21.15.39:0/3420396395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae410b0c0 con 0x7faae4108610 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.437+0000 7faae9c00700 1 --2- 172.21.15.39:0/3420396395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faad80098a0 tx=0x7faad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf1734cc1f94c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.437+0000 7faae3e00700 1 -- 172.21.15.39:0/3420396395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad8004160 con 0x7faae4108610 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.437+0000 7faae3e00700 1 -- 172.21.15.39:0/3420396395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faad8004300 con 0x7faae4108610 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.437+0000 7faae3e00700 1 -- 172.21.15.39:0/3420396395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad8011b50 con 0x7faae4108610 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 -- 172.21.15.39:0/3420396395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 msgr2=0x7faae410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 --2- 172.21.15.39:0/3420396395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faad80098a0 tx=0x7faad8009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 -- 172.21.15.39:0/3420396395 shutdown_connections 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 --2- 172.21.15.39:0/3420396395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae410aa00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 -- 172.21.15.39:0/3420396395 >> 172.21.15.39:0/3420396395 conn(0x7faae4102000 msgr2=0x7faae4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 -- 172.21.15.39:0/3420396395 shutdown_connections 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.438+0000 7faaec600700 1 -- 172.21.15.39:0/3420396395 wait complete. 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.439+0000 7faaec600700 1 Processor -- start 2024-03-16T09:57:15.270 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.439+0000 7faaec600700 1 -- start start 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faaec600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faaec600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faae410c080 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faae9c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faae9c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44264/0 (socket says 172.21.15.39:44264) 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faae9c00700 1 -- 172.21.15.39:0/328904865 learned_addr learned my addr 172.21.15.39:0/328904865 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.440+0000 7faae9c00700 1 -- 172.21.15.39:0/328904865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae41083d0 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faae9c00700 1 --2- 172.21.15.39:0/328904865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faad8011b60 tx=0x7faad8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad8003970 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faad8003b10 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae41065c0 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad8024610 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.441+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae4106b00 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.442+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faad8024c50 con 0x7faae4108610 2024-03-16T09:57:15.271 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.442+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faad8011e00 con 0x7faae4108610 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.442+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae4063900 con 0x7faae4108610 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.445+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7faad801a9f0 con 0x7faae4108610 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.525+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7faae4107120 con 0x7faae4108610 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.526+0000 7faae2000700 1 -- 172.21.15.39:0/328904865 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7faad801a430 con 0x7faae4108610 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.527+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 msgr2=0x7faae4107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.527+0000 7faaec600700 1 --2- 172.21.15.39:0/328904865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faad8011b60 tx=0x7faad8011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.527+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 shutdown_connections 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.528+0000 7faaec600700 1 --2- 172.21.15.39:0/328904865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faae4108610 0x7faae4107e90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.528+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 >> 172.21.15.39:0/328904865 conn(0x7faae4102000 msgr2=0x7faae4102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.528+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 shutdown_connections 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:14.528+0000 7faaec600700 1 -- 172.21.15.39:0/328904865 wait complete. 2024-03-16T09:57:15.272 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2024-03-16T09:57:15.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:15.458+0000 7f590acca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:57:15.897 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:15.584+0000 7f590acca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:57:15.898 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:15.766+0000 7f590acca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:57:15.898 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:15.895+0000 7f590acca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:57:16.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.008+0000 7f590acca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:57:16.614 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.350+0000 7f590acca200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:57:16.615 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.496+0000 7f590acca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:57:16.964 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.612+0000 7f590acca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:57:16.964 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.740+0000 7f590acca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:57:16.964 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:16.962+0000 7f590acca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:57:18.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:17 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4189563006' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:57:18.563 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:18.337+0000 7f590acca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:57:18.563 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:18.515+0000 7f590acca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:57:18.570 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:18.574 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:57:18.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:57:18.575 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:57:18.583 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:57:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:57:03.275227+0000", 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.828+0000 7f457e800700 1 Processor -- start 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.828+0000 7f457e800700 1 -- start start 2024-03-16T09:57:18.585 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.829+0000 7f457e800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.829+0000 7f457e800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f457810af80 con 0x7f4578108650 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.829+0000 7f4577400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.829+0000 7f4577400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44276/0 (socket says 172.21.15.39:44276) 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.829+0000 7f4577400700 1 -- 172.21.15.39:0/3360284406 learned_addr learned my addr 172.21.15.39:0/3360284406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.830+0000 7f4577400700 1 -- 172.21.15.39:0/3360284406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457810b100 con 0x7f4578108650 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.830+0000 7f4577400700 1 --2- 172.21.15.39:0/3360284406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f45680098a0 tx=0x7f4568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f23930074838a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.830+0000 7f4576000700 1 -- 172.21.15.39:0/3360284406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4568004160 con 0x7f4578108650 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.830+0000 7f4576000700 1 -- 172.21.15.39:0/3360284406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4568004300 con 0x7f4578108650 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.830+0000 7f4576000700 1 -- 172.21.15.39:0/3360284406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4568011b50 con 0x7f4578108650 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 -- 172.21.15.39:0/3360284406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 msgr2=0x7f457810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 --2- 172.21.15.39:0/3360284406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f45680098a0 tx=0x7f4568009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 -- 172.21.15.39:0/3360284406 shutdown_connections 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 --2- 172.21.15.39:0/3360284406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457810aa40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:18.586 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 -- 172.21.15.39:0/3360284406 >> 172.21.15.39:0/3360284406 conn(0x7f4578102000 msgr2=0x7f4578104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 -- 172.21.15.39:0/3360284406 shutdown_connections 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.831+0000 7f457e800700 1 -- 172.21.15.39:0/3360284406 wait complete. 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.832+0000 7f457e800700 1 Processor -- start 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.832+0000 7f457e800700 1 -- start start 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f457e800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f457e800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f457810c070 con 0x7f4578108650 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f4577400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f4577400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44290/0 (socket says 172.21.15.39:44290) 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f4577400700 1 -- 172.21.15.39:0/4189563006 learned_addr learned my addr 172.21.15.39:0/4189563006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.833+0000 7f4577400700 1 -- 172.21.15.39:0/4189563006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457819c1b0 con 0x7f4578108650 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f4577400700 1 --2- 172.21.15.39:0/4189563006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4568011b60 tx=0x7f4568011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4568003970 con 0x7f4578108650 2024-03-16T09:57:18.587 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4568003b10 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f457e800700 1 -- 172.21.15.39:0/4189563006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457819c440 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4568024610 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f457e800700 1 -- 172.21.15.39:0/4189563006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457819f130 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.834+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4568024c50 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.835+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4568011e00 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.835+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4550005330 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.838+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4568043dd0 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.915+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f45500058f0 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.915+0000 7f456fe00700 1 -- 172.21.15.39:0/4189563006 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4568032030 con 0x7f4578108650 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.916+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 msgr2=0x7f457819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.916+0000 7f456d600700 1 --2- 172.21.15.39:0/4189563006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4568011b60 tx=0x7f4568011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.916+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 shutdown_connections 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.916+0000 7f456d600700 1 --2- 172.21.15.39:0/4189563006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4578108650 0x7f457819bc70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.917+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 >> 172.21.15.39:0/4189563006 conn(0x7f4578102000 msgr2=0x7f4578102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:18.588 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.917+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 shutdown_connections 2024-03-16T09:57:18.589 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:17.917+0000 7f456d600700 1 -- 172.21.15.39:0/4189563006 wait complete. 2024-03-16T09:57:18.589 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2024-03-16T09:57:19.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:18.942+0000 7f590acca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:57:19.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:19.064+0000 7f590acca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:57:19.586 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:19.330+0000 7f590acca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:57:19.972 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:19.584+0000 7f590acca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:57:19.972 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:19.969+0000 7f590acca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:57:20.610 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:20.292+0000 7f590acca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:57:20.879 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:20.608+0000 7f590acca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:57:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:21 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3008777838' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:57:21.886 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:21 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:21.708+0000 7f590acca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:57:21.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:57:21.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:57:21.908 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:21.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:57:21.910 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:21.915 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:57:03.275227+0000", 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.160+0000 7fd99c600700 1 Processor -- start 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.160+0000 7fd99c600700 1 -- start start 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.161+0000 7fd99c600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.161+0000 7fd99c600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd99407c870 con 0x7fd99407de40 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.161+0000 7fd999c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.161+0000 7fd999c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57484/0 (socket says 172.21.15.39:57484) 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.161+0000 7fd999c00700 1 -- 172.21.15.39:0/1869057978 learned_addr learned my addr 172.21.15.39:0/1869057978 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:21.916 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.162+0000 7fd999c00700 1 -- 172.21.15.39:0/1869057978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99407c9f0 con 0x7fd99407de40 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.162+0000 7fd999c00700 1 --2- 172.21.15.39:0/1869057978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae14d88b7cd8ba94 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.162+0000 7fd993e00700 1 -- 172.21.15.39:0/1869057978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd988004160 con 0x7fd99407de40 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd993e00700 1 -- 172.21.15.39:0/1869057978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd988004300 con 0x7fd99407de40 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd993e00700 1 -- 172.21.15.39:0/1869057978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd988011b50 con 0x7fd99407de40 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd99c600700 1 -- 172.21.15.39:0/1869057978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 msgr2=0x7fd99407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd99c600700 1 --2- 172.21.15.39:0/1869057978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd99c600700 1 -- 172.21.15.39:0/1869057978 shutdown_connections 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd99c600700 1 --2- 172.21.15.39:0/1869057978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99407c2a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.163+0000 7fd99c600700 1 -- 172.21.15.39:0/1869057978 >> 172.21.15.39:0/1869057978 conn(0x7fd994103100 msgr2=0x7fd994105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.164+0000 7fd99c600700 1 -- 172.21.15.39:0/1869057978 shutdown_connections 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.164+0000 7fd99c600700 1 -- 172.21.15.39:0/1869057978 wait complete. 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd99c600700 1 Processor -- start 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd99c600700 1 -- start start 2024-03-16T09:57:21.917 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd99c600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd99c600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd99410e1b0 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd999c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd999c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57498/0 (socket says 172.21.15.39:57498) 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.165+0000 7fd999c00700 1 -- 172.21.15.39:0/3008777838 learned_addr learned my addr 172.21.15.39:0/3008777838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd999c00700 1 -- 172.21.15.39:0/3008777838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99419e2d0 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd999c00700 1 --2- 172.21.15.39:0/3008777838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fb0 tx=0x7fd988011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9880038b0 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd988011e50 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd988019420 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.166+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99419e560 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.167+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a1210 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.167+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd988019a60 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.167+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd98802dc60 con 0x7fd99407de40 2024-03-16T09:57:21.918 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.168+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd994063900 con 0x7fd99407de40 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.170+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd988043dd0 con 0x7fd99407de40 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.250+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd9941a17e0 con 0x7fd99407de40 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.250+0000 7fd992000700 1 -- 172.21.15.39:0/3008777838 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd988032020 con 0x7fd99407de40 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 msgr2=0x7fd99419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 --2- 172.21.15.39:0/3008777838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd988009fb0 tx=0x7fd988011bf0 comp rx=0 tx=0).stop 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 shutdown_connections 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 --2- 172.21.15.39:0/3008777838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd99407de40 0x7fd99419dd90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 >> 172.21.15.39:0/3008777838 conn(0x7fd994103100 msgr2=0x7fd994103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 shutdown_connections 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:21.252+0000 7fd99c600700 1 -- 172.21.15.39:0/3008777838 wait complete. 2024-03-16T09:57:21.919 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (3/15)... 2024-03-16T09:57:22.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:22.433+0000 7f590acca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:57:22.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:22.567+0000 7f590acca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: Activating manager daemon x 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: mgrmap e2: x(active, starting, since 0.011822s) 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: Manager daemon x is now available 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:57:23.159 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:22.859+0000 7f590acca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:57:24.216 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:23 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' 2024-03-16T09:57:24.216 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:23 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' 2024-03-16T09:57:24.216 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:23 smithi039 ceph-mon[31510]: from='mgr.14100 172.21.15.39:0/1024678638' entity='mgr.x' 2024-03-16T09:57:24.216 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:23 smithi039 ceph-mon[31510]: mgrmap e3: x(active, since 1.02215s) 2024-03-16T09:57:25.212 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:24 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2047331227' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:57:25.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:57:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:57:25.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:57:03.275227+0000", 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.531+0000 7f5360200700 1 Processor -- start 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f5360200700 1 -- start start 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f5360200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f5360200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5358108570 con 0x7f5358107b80 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f535d800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:25.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f535d800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57580/0 (socket says 172.21.15.39:57580) 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.532+0000 7f535d800700 1 -- 172.21.15.39:0/2537232875 learned_addr learned my addr 172.21.15.39:0/2537232875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.533+0000 7f535d800700 1 -- 172.21.15.39:0/2537232875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5358108d80 con 0x7f5358107b80 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.533+0000 7f535d800700 1 --2- 172.21.15.39:0/2537232875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f534c009e80 tx=0x7f534c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d809b1c56cf79 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.533+0000 7f5357e00700 1 -- 172.21.15.39:0/2537232875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f534c017070 con 0x7f5358107b80 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.533+0000 7f5357e00700 1 -- 172.21.15.39:0/2537232875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f534c00b8c0 con 0x7f5358107b80 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.533+0000 7f5357e00700 1 -- 172.21.15.39:0/2537232875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f534c0045c0 con 0x7f5358107b80 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 -- 172.21.15.39:0/2537232875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 msgr2=0x7f5358107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 --2- 172.21.15.39:0/2537232875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f534c009e80 tx=0x7f534c00b330 comp rx=0 tx=0).stop 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 -- 172.21.15.39:0/2537232875 shutdown_connections 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 --2- 172.21.15.39:0/2537232875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f5358107fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 -- 172.21.15.39:0/2537232875 >> 172.21.15.39:0/2537232875 conn(0x7f5358103100 msgr2=0x7f5358105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 -- 172.21.15.39:0/2537232875 shutdown_connections 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.534+0000 7f5360200700 1 -- 172.21.15.39:0/2537232875 wait complete. 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.535+0000 7f5360200700 1 Processor -- start 2024-03-16T09:57:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f5360200700 1 -- start start 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f5360200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f5360200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f535810c130 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f535d800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f535d800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57596/0 (socket says 172.21.15.39:57596) 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.536+0000 7f535d800700 1 -- 172.21.15.39:0/2047331227 learned_addr learned my addr 172.21.15.39:0/2047331227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f535d800700 1 -- 172.21.15.39:0/2047331227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535819e310 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f535d800700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f534c009e50 tx=0x7f534c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f534c020020 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f534c003b50 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535819e5a0 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f534c017040 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.537+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53581a1290 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.538+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5358063900 con 0x7f5358107b80 2024-03-16T09:57:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.539+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 3) ==== 45989+0+0 (secure 0 0 0) 0x7f534c023020 con 0x7f5358107b80 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.539+0000 7f5356000700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 0x7f534003b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.539+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f534c00f050 con 0x7f5358107b80 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.540+0000 7f535ce00700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 0x7f534003b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.540+0000 7f535ce00700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 0x7f534003b260 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5344006fd0 tx=0x7f5344006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.546+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f534c02a020 con 0x7f5358107b80 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.855+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f53581a1960 con 0x7f5358107b80 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.856+0000 7f5356000700 1 -- 172.21.15.39:0/2047331227 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f534c003d00 con 0x7f5358107b80 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 msgr2=0x7f534003b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 0x7f534003b260 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5344006fd0 tx=0x7f5344006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 msgr2=0x7f535819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f534c009e50 tx=0x7f534c00bd30 comp rx=0 tx=0).stop 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 shutdown_connections 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f5340038da0 0x7f534003b260 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:25.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 --2- 172.21.15.39:0/2047331227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5358107b80 0x7f535819ddd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:25.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.859+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 >> 172.21.15.39:0/2047331227 conn(0x7f5358103100 msgr2=0x7f5358103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:25.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.860+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 shutdown_connections 2024-03-16T09:57:25.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:24.860+0000 7f5360200700 1 -- 172.21.15.39:0/2047331227 wait complete. 2024-03-16T09:57:25.558 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2024-03-16T09:57:26.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:25 smithi039 ceph-mon[31510]: mgrmap e4: x(active, since 2s) 2024-03-16T09:57:27.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3260716565' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T09:57:27.055 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:27.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T09:57:27.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:27.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T09:57:27.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T09:57:27.057 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T09:57:27.057 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T09:57:27.057 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:27.057 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T09:57:27.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T09:57:27.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-03-16T09:57:27.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T09:57:27.058 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T09:57:27.059 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T09:57:27.059 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T09:57:27.059 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T09:57:27.059 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.135+0000 7f107c400700 1 Processor -- start 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.136+0000 7f107c400700 1 -- start start 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.136+0000 7f107c400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.136+0000 7f107c400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10740823f0 con 0x7f107410ec90 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.136+0000 7f1079a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.137+0000 7f1079a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57610/0 (socket says 172.21.15.39:57610) 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.137+0000 7f1079a00700 1 -- 172.21.15.39:0/29002624 learned_addr learned my addr 172.21.15.39:0/29002624 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.137+0000 7f1079a00700 1 -- 172.21.15.39:0/29002624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10741115c0 con 0x7f107410ec90 2024-03-16T09:57:27.061 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.137+0000 7f1079a00700 1 --2- 172.21.15.39:0/29002624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f10680098a0 tx=0x7f1068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538b35f162d9889a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f1073e00700 1 -- 172.21.15.39:0/29002624 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1068004160 con 0x7f107410ec90 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f1073e00700 1 -- 172.21.15.39:0/29002624 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1068004300 con 0x7f107410ec90 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f1073e00700 1 -- 172.21.15.39:0/29002624 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1068011b50 con 0x7f107410ec90 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f107c400700 1 -- 172.21.15.39:0/29002624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 msgr2=0x7f1074111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f107c400700 1 --2- 172.21.15.39:0/29002624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f10680098a0 tx=0x7f1068009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f107c400700 1 -- 172.21.15.39:0/29002624 shutdown_connections 2024-03-16T09:57:27.064 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f107c400700 1 --2- 172.21.15.39:0/29002624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f1074111080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.138+0000 7f107c400700 1 -- 172.21.15.39:0/29002624 >> 172.21.15.39:0/29002624 conn(0x7f107407d1b0 msgr2=0x7f107407f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.139+0000 7f107c400700 1 -- 172.21.15.39:0/29002624 shutdown_connections 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.139+0000 7f107c400700 1 -- 172.21.15.39:0/29002624 wait complete. 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.140+0000 7f107c400700 1 Processor -- start 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.140+0000 7f107c400700 1 -- start start 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.140+0000 7f107c400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.140+0000 7f107c400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1074112580 con 0x7f107410ec90 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.140+0000 7f1079a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1079a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57620/0 (socket says 172.21.15.39:57620) 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1079a00700 1 -- 172.21.15.39:0/3260716565 learned_addr learned my addr 172.21.15.39:0/3260716565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1079a00700 1 -- 172.21.15.39:0/3260716565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10741a2600 con 0x7f107410ec90 2024-03-16T09:57:27.065 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1079a00700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1068011670 tx=0x7f1068011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:27.066 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10680039c0 con 0x7f107410ec90 2024-03-16T09:57:27.066 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1068011e50 con 0x7f107410ec90 2024-03-16T09:57:27.066 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10741a2890 con 0x7f107410ec90 2024-03-16T09:57:27.068 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.141+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f106801a5c0 con 0x7f107410ec90 2024-03-16T09:57:27.068 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.142+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10741a5580 con 0x7f107410ec90 2024-03-16T09:57:27.068 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.143+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f1068024430 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.143+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1074063900 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.143+0000 7f1072000700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 0x7f105c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.143+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f106804d190 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.144+0000 7f1079000700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 0x7f105c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.144+0000 7f1079000700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 0x7f105c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.149+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f106804cbb0 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.372+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f10741a5750 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.375+0000 7f1072000700 1 -- 172.21.15.39:0/3260716565 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f10741a5750 con 0x7f107410ec90 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 msgr2=0x7f105c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 0x7f105c03b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1060006fd0 tx=0x7f1060006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 msgr2=0x7f10741a20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1068011670 tx=0x7f1068011b60 comp rx=0 tx=0).stop 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 shutdown_connections 2024-03-16T09:57:27.069 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f105c038ed0 0x7f105c03b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:27.070 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 --2- 172.21.15.39:0/3260716565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f107410ec90 0x7f10741a20c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:27.070 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.378+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 >> 172.21.15.39:0/3260716565 conn(0x7f107407d1b0 msgr2=0x7f107407dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:27.070 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.379+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 shutdown_connections 2024-03-16T09:57:27.070 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:26.379+0000 7f107c400700 1 -- 172.21.15.39:0/3260716565 wait complete. 2024-03-16T09:57:27.070 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2024-03-16T09:57:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:28 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2210019242' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-16T09:57:28.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setuser ceph since I am not root 2024-03-16T09:57:28.256 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setgroup ceph since I am not root 2024-03-16T09:57:28.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.653+0000 7ff5e8e00700 1 Processor -- start 2024-03-16T09:57:28.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e8e00700 1 -- start start 2024-03-16T09:57:28.667 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e8e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e8e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40823f0 con 0x7ff5e410ec90 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e1e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e1e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57624/0 (socket says 172.21.15.39:57624) 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.654+0000 7ff5e1e00700 1 -- 172.21.15.39:0/4048133189 learned_addr learned my addr 172.21.15.39:0/4048133189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.655+0000 7ff5e1e00700 1 -- 172.21.15.39:0/4048133189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41115c0 con 0x7ff5e410ec90 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.655+0000 7ff5e1e00700 1 --2- 172.21.15.39:0/4048133189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098a0 tx=0x7ff5d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e948920811b80fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e0a00700 1 -- 172.21.15.39:0/4048133189 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4004160 con 0x7ff5e410ec90 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e0a00700 1 -- 172.21.15.39:0/4048133189 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5d4004300 con 0x7ff5e410ec90 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e0a00700 1 -- 172.21.15.39:0/4048133189 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4011b50 con 0x7ff5e410ec90 2024-03-16T09:57:28.672 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 -- 172.21.15.39:0/4048133189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 msgr2=0x7ff5e4111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/4048133189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff5d40098a0 tx=0x7ff5d4009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 -- 172.21.15.39:0/4048133189 shutdown_connections 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/4048133189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e4111080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 -- 172.21.15.39:0/4048133189 >> 172.21.15.39:0/4048133189 conn(0x7ff5e407d1b0 msgr2=0x7ff5e407f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 -- 172.21.15.39:0/4048133189 shutdown_connections 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.656+0000 7ff5e8e00700 1 -- 172.21.15.39:0/4048133189 wait complete. 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.657+0000 7ff5e8e00700 1 Processor -- start 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e8e00700 1 -- start start 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e8e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e8e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e4112580 con 0x7ff5e410ec90 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e1e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e1e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57638/0 (socket says 172.21.15.39:57638) 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.658+0000 7ff5e1e00700 1 -- 172.21.15.39:0/2210019242 learned_addr learned my addr 172.21.15.39:0/2210019242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:28.673 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5e1e00700 1 -- 172.21.15.39:0/2210019242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e41a2640 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5e1e00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011b60 tx=0x7ff5d4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4003b20 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5d4003cc0 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5d4024840 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.659+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41a28d0 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.660+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e41a55c0 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.661+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e4063900 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.661+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7ff5d4022020 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.661+0000 7ff5daa00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 0x7ff5c803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.661+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5d4011e00 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.662+0000 7ff5e1400700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 0x7ff5c803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.662+0000 7ff5e1400700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 0x7ff5c803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006fd0 tx=0x7ff5d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.668+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff5d404c8e0 con 0x7ff5e410ec90 2024-03-16T09:57:28.674 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:27.950+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff5e41a5790 con 0x7ff5e410ec90 2024-03-16T09:57:28.675 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.008+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff5e41a5790 con 0x7ff5e410ec90 2024-03-16T09:57:28.675 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.011+0000 7ff5daa00700 1 -- 172.21.15.39:0/2210019242 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7ff5d400b3e0 con 0x7ff5e410ec90 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 msgr2=0x7ff5c803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 0x7ff5c803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006fd0 tx=0x7ff5d0006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 msgr2=0x7ff5e41a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4011b60 tx=0x7ff5d4011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 shutdown_connections 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7ff5c8038f20 0x7ff5c803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 --2- 172.21.15.39:0/2210019242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5e410ec90 0x7ff5e41a2100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.012+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 >> 172.21.15.39:0/2210019242 conn(0x7ff5e407d1b0 msgr2=0x7ff5e407dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.013+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 shutdown_connections 2024-03-16T09:57:28.676 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:28.013+0000 7ff5e8e00700 1 -- 172.21.15.39:0/2210019242 wait complete. 2024-03-16T09:57:28.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:28.868+0000 7f8e9e9f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:57:29.247 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:28.996+0000 7f8e9e9f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:57:29.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:29 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2210019242' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-16T09:57:29.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:29 smithi039 ceph-mon[31510]: mgrmap e5: x(active, since 5s) 2024-03-16T09:57:30.138 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2056697176' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T09:57:30.139 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:29.840+0000 7f8e9e9f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:57:30.139 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:29.967+0000 7f8e9e9f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.246+0000 7fb587c00700 1 Processor -- start 2024-03-16T09:57:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.246+0000 7fb587c00700 1 -- start start 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.247+0000 7fb587c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.247+0000 7fb587c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb580108570 con 0x7fb580107b80 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.247+0000 7fb585200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.247+0000 7fb585200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57674/0 (socket says 172.21.15.39:57674) 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.247+0000 7fb585200700 1 -- 172.21.15.39:0/3709448353 learned_addr learned my addr 172.21.15.39:0/3709448353 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.248+0000 7fb585200700 1 -- 172.21.15.39:0/3709448353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580108d80 con 0x7fb580107b80 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.248+0000 7fb585200700 1 --2- 172.21.15.39:0/3709448353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5740098a0 tx=0x7fb574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77a86195c0ceb54 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.248+0000 7fb57f400700 1 -- 172.21.15.39:0/3709448353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574004160 con 0x7fb580107b80 2024-03-16T09:57:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.248+0000 7fb57f400700 1 -- 172.21.15.39:0/3709448353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb574004300 con 0x7fb580107b80 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.248+0000 7fb57f400700 1 -- 172.21.15.39:0/3709448353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574011b50 con 0x7fb580107b80 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 -- 172.21.15.39:0/3709448353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 msgr2=0x7fb580107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 --2- 172.21.15.39:0/3709448353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5740098a0 tx=0x7fb574009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 -- 172.21.15.39:0/3709448353 shutdown_connections 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 --2- 172.21.15.39:0/3709448353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb580107fa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 -- 172.21.15.39:0/3709448353 >> 172.21.15.39:0/3709448353 conn(0x7fb580103100 msgr2=0x7fb580105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 -- 172.21.15.39:0/3709448353 shutdown_connections 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.249+0000 7fb587c00700 1 -- 172.21.15.39:0/3709448353 wait complete. 2024-03-16T09:57:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.250+0000 7fb587c00700 1 Processor -- start 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.250+0000 7fb587c00700 1 -- start start 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb587c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb587c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010c130 con 0x7fb580107b80 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb585200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb585200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57688/0 (socket says 172.21.15.39:57688) 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb585200700 1 -- 172.21.15.39:0/2056697176 learned_addr learned my addr 172.21.15.39:0/2056697176 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.251+0000 7fb585200700 1 -- 172.21.15.39:0/2056697176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58019e340 con 0x7fb580107b80 2024-03-16T09:57:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb585200700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb574011850 tx=0x7fb574011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb574003af0 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb574020070 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5740195a0 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58019e5d0 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.252+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5801a12c0 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.253+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fb574019850 con 0x7fb580107b80 2024-03-16T09:57:30.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.253+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb580063900 con 0x7fb580107b80 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.254+0000 7fb57d600700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 0x7fb56803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.254+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb574011e00 con 0x7fb580107b80 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.254+0000 7fb57fe00700 1 -- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 msgr2=0x7fb56803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.254+0000 7fb57fe00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 0x7fb56803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.259+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb574019b40 con 0x7fb580107b80 2024-03-16T09:57:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.455+0000 7fb57fe00700 1 -- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 msgr2=0x7fb56803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.455+0000 7fb57fe00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 0x7fb56803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.534+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb5801a1490 con 0x7fb580107b80 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.534+0000 7fb57d600700 1 -- 172.21.15.39:0/2056697176 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb5801a1490 con 0x7fb580107b80 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.537+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 msgr2=0x7fb56803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.537+0000 7fb587c00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 0x7fb56803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.537+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 msgr2=0x7fb58019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:30.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.537+0000 7fb587c00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb574011850 tx=0x7fb574011c10 comp rx=0 tx=0).stop 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 shutdown_connections 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7fb568038f70 0x7fb56803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 --2- 172.21.15.39:0/2056697176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb580107b80 0x7fb58019de00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 >> 172.21.15.39:0/2056697176 conn(0x7fb580103100 msgr2=0x7fb580103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 shutdown_connections 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:29.538+0000 7fb587c00700 1 -- 172.21.15.39:0/2056697176 wait complete. 2024-03-16T09:57:30.164 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-03-16T09:57:30.165 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2024-03-16T09:57:30.406 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:30.154+0000 7f8e9e9f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:57:30.406 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:30.286+0000 7f8e9e9f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:57:30.406 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:30.404+0000 7f8e9e9f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:57:31.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:30.757+0000 7f8e9e9f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:57:31.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:30.903+0000 7f8e9e9f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:57:31.309 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:31.020+0000 7f8e9e9f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:57:31.309 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:31.148+0000 7f8e9e9f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:57:31.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:31.373+0000 7f8e9e9f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:57:33.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:32.701+0000 7f8e9e9f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:57:33.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:32.880+0000 7f8e9e9f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:57:33.701 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:33.310+0000 7f8e9e9f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:57:33.701 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:33.432+0000 7f8e9e9f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:57:33.701 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:33.700+0000 7f8e9e9f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:57:34.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:33.954+0000 7f8e9e9f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:57:34.674 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:34.345+0000 7f8e9e9f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:57:34.674 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:34.670+0000 7f8e9e9f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:57:35.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:34.993+0000 7f8e9e9f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:57:36.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:36.077+0000 7f8e9e9f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:57:37.233 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:36.801+0000 7f8e9e9f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:57:37.233 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:36.937+0000 7f8e9e9f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:57:37.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:57:37.232+0000 7f8e9e9f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: Active manager daemon x restarted 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: Activating manager daemon x 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: osdmap e2: 0 total, 0 up, 0 in 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: mgrmap e6: x(active, starting, since 0.0114298s) 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: Manager daemon x is now available 2024-03-16T09:57:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:37 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: Found migration_current of "None". Setting to last migration. 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:57:38.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:38 smithi039 ceph-mon[31510]: mgrmap e7: x(active, since 1.01386s) 2024-03-16T09:57:39.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:57:39.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-16T09:57:39.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T09:57:39.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:57:39.041 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.697+0000 7f7b9a200700 1 Processor -- start 2024-03-16T09:57:39.042 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.698+0000 7f7b9a200700 1 -- start start 2024-03-16T09:57:39.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.698+0000 7f7b9a200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:39.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.698+0000 7f7b9a200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9407bed0 con 0x7f7b941064a0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.699+0000 7f7b93400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.699+0000 7f7b93400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58494/0 (socket says 172.21.15.39:58494) 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.699+0000 7f7b93400700 1 -- 172.21.15.39:0/2917139655 learned_addr learned my addr 172.21.15.39:0/2917139655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.699+0000 7f7b93400700 1 -- 172.21.15.39:0/2917139655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b94106e00 con 0x7f7b941064a0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.700+0000 7f7b93400700 1 --2- 172.21.15.39:0/2917139655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b840098a0 tx=0x7f7b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983352eb4246e23d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.700+0000 7f7b92000700 1 -- 172.21.15.39:0/2917139655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84004160 con 0x7f7b941064a0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.700+0000 7f7b92000700 1 -- 172.21.15.39:0/2917139655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b84004300 con 0x7f7b941064a0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.700+0000 7f7b92000700 1 -- 172.21.15.39:0/2917139655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84011b50 con 0x7f7b941064a0 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 -- 172.21.15.39:0/2917139655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 msgr2=0x7f7b941068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 --2- 172.21.15.39:0/2917139655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b840098a0 tx=0x7f7b84009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 -- 172.21.15.39:0/2917139655 shutdown_connections 2024-03-16T09:57:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 --2- 172.21.15.39:0/2917139655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b941068c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 -- 172.21.15.39:0/2917139655 >> 172.21.15.39:0/2917139655 conn(0x7f7b94102000 msgr2=0x7f7b94104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 -- 172.21.15.39:0/2917139655 shutdown_connections 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.701+0000 7f7b9a200700 1 -- 172.21.15.39:0/2917139655 wait complete. 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.702+0000 7f7b9a200700 1 Processor -- start 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b9a200700 1 -- start start 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b9a200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:39.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b9a200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b94107e90 con 0x7f7b941064a0 2024-03-16T09:57:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b93400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b93400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58500/0 (socket says 172.21.15.39:58500) 2024-03-16T09:57:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.703+0000 7f7b93400700 1 -- 172.21.15.39:0/699453821 learned_addr learned my addr 172.21.15.39:0/699453821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b93400700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9419e3a0 con 0x7f7b941064a0 2024-03-16T09:57:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b93400700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b84011b60 tx=0x7f7b84011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84003970 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b84003b10 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9419e630 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84024610 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.704+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941a1320 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f7b8401a9f0 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b8be00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b7803baf0 con 0x7f7b78038f20 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b84011e00 con 0x7f7b941064a0 2024-03-16T09:57:39.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.706+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.907+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:30.907+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:31.308+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:31.308+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:32.109+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:32.109+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:33.711+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:33.711+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:36.915+0000 7f7b92a00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/3969870217 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:36.915+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:37.243+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7f7b84024c90 con 0x7f7b941064a0 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:37.243+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 msgr2=0x7f7b7803b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:57:39.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:37.243+0000 7f7b8be00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.247+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f7b8404ed60 con 0x7f7b941064a0 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.247+0000 7f7b8be00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 0x7f7b7803ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.248+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b84012ea0 con 0x7f7b7803c810 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.250+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 0x7f7b7803ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.251+0000 7f7b92a00700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 0x7f7b7803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b80003a10 tx=0x7f7b80009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.253+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7b84012ea0 con 0x7f7b7803c810 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.262+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7b9419eb10 con 0x7f7b7803c810 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.262+0000 7f7b8be00700 1 -- 172.21.15.39:0/699453821 <== mgr.14120 v2:172.21.15.39:6800/1974903299 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7b9419eb10 con 0x7f7b7803c810 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 msgr2=0x7f7b7803ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 0x7f7b7803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b80003a10 tx=0x7f7b80009290 comp rx=0 tx=0).stop 2024-03-16T09:57:39.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 msgr2=0x7f7b9419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b84011b60 tx=0x7f7b84011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 shutdown_connections 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7b7803c810 0x7f7b7803ec00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b941064a0 0x7f7b9419de60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 --2- 172.21.15.39:0/699453821 >> [v2:172.21.15.39:6800/3969870217,v1:172.21.15.39:6801/3969870217] conn(0x7f7b78038f20 0x7f7b7803b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 >> 172.21.15.39:0/699453821 conn(0x7f7b94102000 msgr2=0x7f7b94194aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 shutdown_connections 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:38.263+0000 7f7b9a200700 1 -- 172.21.15.39:0/699453821 wait complete. 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2024-03-16T09:57:39.056 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2024-03-16T09:57:39.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:39 smithi039 ceph-mon[31510]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T09:57:39.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:39 smithi039 ceph-mon[31510]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T09:57:39.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:39 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:39.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:39 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: [16/Mar/2024:09:57:39] ENGINE Bus STARTING 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: mgrmap e8: x(active, since 2s) 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: [16/Mar/2024:09:57:39] ENGINE Serving on http://172.21.15.39:8765 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: [16/Mar/2024:09:57:39] ENGINE Serving on https://172.21.15.39:7150 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: [16/Mar/2024:09:57:39] ENGINE Bus STARTED 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:40.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:40 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.596+0000 7f97bba00700 1 Processor -- start 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.596+0000 7f97bba00700 1 -- start start 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.596+0000 7f97bba00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.596+0000 7f97bba00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410a7a0 con 0x7f97b4109db0 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.597+0000 7f97b9000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:40.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.597+0000 7f97b9000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58568/0 (socket says 172.21.15.39:58568) 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.597+0000 7f97b9000700 1 -- 172.21.15.39:0/1039167932 learned_addr learned my addr 172.21.15.39:0/1039167932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.597+0000 7f97b9000700 1 -- 172.21.15.39:0/1039167932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b410afb0 con 0x7f97b4109db0 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.598+0000 7f97b9000700 1 --2- 172.21.15.39:0/1039167932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a563c197daf2b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.598+0000 7f97b3400700 1 -- 172.21.15.39:0/1039167932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97a8004160 con 0x7f97b4109db0 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.598+0000 7f97b3400700 1 -- 172.21.15.39:0/1039167932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97a8004300 con 0x7f97b4109db0 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.598+0000 7f97b3400700 1 -- 172.21.15.39:0/1039167932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97a8011b50 con 0x7f97b4109db0 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 -- 172.21.15.39:0/1039167932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 msgr2=0x7f97b410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 --2- 172.21.15.39:0/1039167932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 -- 172.21.15.39:0/1039167932 shutdown_connections 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 --2- 172.21.15.39:0/1039167932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b410a1d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 -- 172.21.15.39:0/1039167932 >> 172.21.15.39:0/1039167932 conn(0x7f97b407d150 msgr2=0x7f97b407d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 -- 172.21.15.39:0/1039167932 shutdown_connections 2024-03-16T09:57:40.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.599+0000 7f97bba00700 1 -- 172.21.15.39:0/1039167932 wait complete. 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.600+0000 7f97bba00700 1 Processor -- start 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.600+0000 7f97bba00700 1 -- start start 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97bba00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97bba00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410e360 con 0x7f97b4109db0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97b9000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97b9000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58582/0 (socket says 172.21.15.39:58582) 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97b9000700 1 -- 172.21.15.39:0/3210771254 learned_addr learned my addr 172.21.15.39:0/3210771254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.601+0000 7f97b9000700 1 -- 172.21.15.39:0/3210771254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b419e310 con 0x7f97b4109db0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97b9000700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f97a8011850 tx=0x7f97a8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97a8003af0 con 0x7f97b4109db0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97a8020070 con 0x7f97b4109db0 2024-03-16T09:57:40.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97a80195a0 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97bba00700 1 -- 172.21.15.39:0/3210771254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b419e5a0 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.602+0000 7f97bba00700 1 -- 172.21.15.39:0/3210771254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a1290 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.603+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f97a8019850 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.604+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b4063900 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.604+0000 7f97b1600700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 0x7f979c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.604+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f97a8011e00 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.604+0000 7f97b3e00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 0x7f979c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.605+0000 7f97b3e00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 0x7f979c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.611+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f97a802b260 con 0x7f97b4109db0 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.853+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f97b4063070 con 0x7f979c038f70 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.869+0000 7f97b1600700 1 -- 172.21.15.39:0/3210771254 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97b4063070 con 0x7f979c038f70 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 msgr2=0x7f979c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:40.618 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 0x7f979c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97a0006fd0 tx=0x7f97a0006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 msgr2=0x7f97b419ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f97a8011850 tx=0x7f97a8011c10 comp rx=0 tx=0).stop 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 shutdown_connections 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f979c038f70 0x7f979c03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 --2- 172.21.15.39:0/3210771254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97b4109db0 0x7f97b419ddd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.872+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 >> 172.21.15.39:0/3210771254 conn(0x7f97b407d150 msgr2=0x7f97b41074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.873+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 shutdown_connections 2024-03-16T09:57:40.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:39.873+0000 7f97a6a00700 1 -- 172.21.15.39:0/3210771254 wait complete. 2024-03-16T09:57:42.098 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.172+0000 7fdce3000700 1 Processor -- start 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.172+0000 7fdce3000700 1 -- start start 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.173+0000 7fdce3000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.173+0000 7fdce3000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc106d60 con 0x7fdcdc106400 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.173+0000 7fdcdbe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.173+0000 7fdcdbe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38674/0 (socket says 172.21.15.39:38674) 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.173+0000 7fdcdbe00700 1 -- 172.21.15.39:0/1482693512 learned_addr learned my addr 172.21.15.39:0/1482693512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.174+0000 7fdcdbe00700 1 -- 172.21.15.39:0/1482693512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc106ee0 con 0x7fdcdc106400 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.174+0000 7fdcdbe00700 1 --2- 172.21.15.39:0/1482693512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788ad11de12a768 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:42.100 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.174+0000 7fdcdaa00700 1 -- 172.21.15.39:0/1482693512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc004160 con 0x7fdcdc106400 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.174+0000 7fdcdaa00700 1 -- 172.21.15.39:0/1482693512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdccc004300 con 0x7fdcdc106400 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.174+0000 7fdcdaa00700 1 -- 172.21.15.39:0/1482693512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc011b50 con 0x7fdcdc106400 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 -- 172.21.15.39:0/1482693512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 msgr2=0x7fdcdc106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 --2- 172.21.15.39:0/1482693512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0098a0 tx=0x7fdccc009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 -- 172.21.15.39:0/1482693512 shutdown_connections 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 --2- 172.21.15.39:0/1482693512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc106820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 -- 172.21.15.39:0/1482693512 >> 172.21.15.39:0/1482693512 conn(0x7fdcdc102000 msgr2=0x7fdcdc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 -- 172.21.15.39:0/1482693512 shutdown_connections 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.175+0000 7fdce3000700 1 -- 172.21.15.39:0/1482693512 wait complete. 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.176+0000 7fdce3000700 1 Processor -- start 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.176+0000 7fdce3000700 1 -- start start 2024-03-16T09:57:42.103 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdce3000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:42.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdce3000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc107f70 con 0x7fdcdc106400 2024-03-16T09:57:42.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdcdbe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:42.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdcdbe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38676/0 (socket says 172.21.15.39:38676) 2024-03-16T09:57:42.104 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdcdbe00700 1 -- 172.21.15.39:0/730219003 learned_addr learned my addr 172.21.15.39:0/730219003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.177+0000 7fdcdbe00700 1 -- 172.21.15.39:0/730219003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc19e3b0 con 0x7fdcdc106400 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdcdbe00700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdccc011b60 tx=0x7fdccc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc003970 con 0x7fdcdc106400 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdccc003b10 con 0x7fdcdc106400 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc19e640 con 0x7fdcdc106400 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc024610 con 0x7fdcdc106400 2024-03-16T09:57:42.107 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.178+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a1330 con 0x7fdcdc106400 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.179+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fdccc01a9f0 con 0x7fdcdc106400 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.179+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc063900 con 0x7fdcdc106400 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.180+0000 7fdcd8c00700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 0x7fdcc003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.180+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdccc011e00 con 0x7fdcdc106400 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.180+0000 7fdcdb400700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 0x7fdcc003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.181+0000 7fdcdb400700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 0x7fdcc003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.186+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdccc019e10 con 0x7fdcdc106400 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.417+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fdcdc02dda0 con 0x7fdcc0038f70 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.419+0000 7fdcd8c00700 1 -- 172.21.15.39:0/730219003 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fdcdc02dda0 con 0x7fdcc0038f70 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.422+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 msgr2=0x7fdcc003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.422+0000 7fdce3000700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 0x7fdcc003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 msgr2=0x7fdcdc19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdccc011b60 tx=0x7fdccc011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:42.108 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 shutdown_connections 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fdcc0038f70 0x7fdcc003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 --2- 172.21.15.39:0/730219003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc106400 0x7fdcdc19de70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 >> 172.21.15.39:0/730219003 conn(0x7fdcdc102000 msgr2=0x7fdcdc194a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 shutdown_connections 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:41.423+0000 7fdce3000700 1 -- 172.21.15.39:0/730219003 wait complete. 2024-03-16T09:57:42.109 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2024-03-16T09:57:42.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:42 smithi039 ceph-mon[31510]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:43.456 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: Generating public/private rsa key pair. 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: Your identification has been saved in /tmp/tmpyxatw6go/key. 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: Your public key has been saved in /tmp/tmpyxatw6go/key.pub. 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: The key fingerprint is: 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: SHA256:qw8gJrB2YXnpy3MEEJngFLC0mq6bbG3nc7emOVT4ZdQ ceph-65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: The key's randomart image is: 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: +---[RSA 3072]----+ 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |o=oo+ . | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |+..oo . . E | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |oo + + . . | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |oo. + .. . o | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |=.o... .S o | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |oo....o. o | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: | .. +o.. | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |oo o oo=.o | 2024-03-16T09:57:43.457 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: |=o. o.++*.. | 2024-03-16T09:57:43.458 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:57:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: +----[SHA256]-----+ 2024-03-16T09:57:43.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.720+0000 7f7f52800700 1 Processor -- start 2024-03-16T09:57:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.720+0000 7f7f52800700 1 -- start start 2024-03-16T09:57:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.721+0000 7f7f52800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.721+0000 7f7f52800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c07c870 con 0x7f7f4c07de40 2024-03-16T09:57:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.721+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.721+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38682/0 (socket says 172.21.15.39:38682) 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.721+0000 7f7f4b400700 1 -- 172.21.15.39:0/3624005056 learned_addr learned my addr 172.21.15.39:0/3624005056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.722+0000 7f7f4b400700 1 -- 172.21.15.39:0/3624005056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c07c9f0 con 0x7f7f4c07de40 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.722+0000 7f7f4b400700 1 --2- 172.21.15.39:0/3624005056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0098a0 tx=0x7f7f3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa84773b1371ba8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f4a000700 1 -- 172.21.15.39:0/3624005056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c004160 con 0x7f7f4c07de40 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f4a000700 1 -- 172.21.15.39:0/3624005056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c004300 con 0x7f7f4c07de40 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f4a000700 1 -- 172.21.15.39:0/3624005056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c011b50 con 0x7f7f4c07de40 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f52800700 1 -- 172.21.15.39:0/3624005056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 msgr2=0x7f7f4c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f52800700 1 --2- 172.21.15.39:0/3624005056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0098a0 tx=0x7f7f3c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f52800700 1 -- 172.21.15.39:0/3624005056 shutdown_connections 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f52800700 1 --2- 172.21.15.39:0/3624005056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c07c2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.723+0000 7f7f52800700 1 -- 172.21.15.39:0/3624005056 >> 172.21.15.39:0/3624005056 conn(0x7f7f4c103100 msgr2=0x7f7f4c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:43.767 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.724+0000 7f7f52800700 1 -- 172.21.15.39:0/3624005056 shutdown_connections 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.724+0000 7f7f52800700 1 -- 172.21.15.39:0/3624005056 wait complete. 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f52800700 1 Processor -- start 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f52800700 1 -- start start 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f52800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f52800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c10e1b0 con 0x7f7f4c07de40 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f4b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38698/0 (socket says 172.21.15.39:38698) 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.725+0000 7f7f4b400700 1 -- 172.21.15.39:0/1135112212 learned_addr learned my addr 172.21.15.39:0/1135112212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f4b400700 1 -- 172.21.15.39:0/1135112212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19e2d0 con 0x7f7f4c07de40 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f4b400700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c011b60 tx=0x7f7f3c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:43.768 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c003970 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c003b10 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c024610 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19e560 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.726+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a1210 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.727+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c063900 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.728+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f7f3c01a430 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.728+0000 7f7f43e00700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 0x7f7f3003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.728+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f3c011e00 con 0x7f7f4c07de40 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.729+0000 7f7f4aa00700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 0x7f7f3003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.729+0000 7f7f4aa00700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 0x7f7f3003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:43.769 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.734+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f3c00e260 con 0x7f7f4c07de40 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:42.950+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7f4c1043f0 con 0x7f7f30038f70 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.140+0000 7f7f43e00700 1 -- 172.21.15.39:0/1135112212 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f4c1043f0 con 0x7f7f30038f70 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 msgr2=0x7f7f3003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 0x7f7f3003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f38006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 msgr2=0x7f7f4c19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c011b60 tx=0x7f7f3c011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 shutdown_connections 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f7f30038f70 0x7f7f3003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 --2- 172.21.15.39:0/1135112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f4c07de40 0x7f7f4c19dd90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 >> 172.21.15.39:0/1135112212 conn(0x7f7f4c103100 msgr2=0x7f7f4c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:43.770 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 shutdown_connections 2024-03-16T09:57:43.771 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:43.143+0000 7f7f52800700 1 -- 172.21.15.39:0/1135112212 wait complete. 2024-03-16T09:57:44.361 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:44 smithi039 ceph-mon[31510]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:44.362 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:44 smithi039 ceph-mon[31510]: Generating ssh key... 2024-03-16T09:57:44.362 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:44 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:44.362 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:44 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:45.356 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:45 smithi039 ceph-mon[31510]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:45.376 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.360+0000 7f4bd0000700 1 Processor -- start 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.360+0000 7f4bd0000700 1 -- start start 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.361+0000 7f4bd0000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.361+0000 7f4bd0000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8108570 con 0x7f4bc8107b80 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.361+0000 7f4bcd600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.361+0000 7f4bcd600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38704/0 (socket says 172.21.15.39:38704) 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.361+0000 7f4bcd600700 1 -- 172.21.15.39:0/273242434 learned_addr learned my addr 172.21.15.39:0/273242434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.362+0000 7f4bcd600700 1 -- 172.21.15.39:0/273242434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc8108d80 con 0x7f4bc8107b80 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.362+0000 7f4bcd600700 1 --2- 172.21.15.39:0/273242434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c44abc92fc2d7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.362+0000 7f4bc7e00700 1 -- 172.21.15.39:0/273242434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc004160 con 0x7f4bc8107b80 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bc7e00700 1 -- 172.21.15.39:0/273242434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bbc004300 con 0x7f4bc8107b80 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bc7e00700 1 -- 172.21.15.39:0/273242434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc011b50 con 0x7f4bc8107b80 2024-03-16T09:57:45.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 -- 172.21.15.39:0/273242434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 msgr2=0x7f4bc8107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 --2- 172.21.15.39:0/273242434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 -- 172.21.15.39:0/273242434 shutdown_connections 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 --2- 172.21.15.39:0/273242434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc8107fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 -- 172.21.15.39:0/273242434 >> 172.21.15.39:0/273242434 conn(0x7f4bc8103100 msgr2=0x7f4bc8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.363+0000 7f4bd0000700 1 -- 172.21.15.39:0/273242434 shutdown_connections 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.364+0000 7f4bd0000700 1 -- 172.21.15.39:0/273242434 wait complete. 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bd0000700 1 Processor -- start 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bd0000700 1 -- start start 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bd0000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bd0000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc810c130 con 0x7f4bc8107b80 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bcd600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bcd600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38716/0 (socket says 172.21.15.39:38716) 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.365+0000 7f4bcd600700 1 -- 172.21.15.39:0/3868585688 learned_addr learned my addr 172.21.15.39:0/3868585688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:45.378 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bcd600700 1 -- 172.21.15.39:0/3868585688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc819e340 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bcd600700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc011850 tx=0x7f4bbc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc003af0 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bbc020070 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc819e5d0 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.366+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc0195a0 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.367+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc81a12c0 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.368+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc8063900 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.368+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f4bbc0198a0 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.368+0000 7f4bc6000700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 0x7f4bb003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.368+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4bbc011e00 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.369+0000 7f4bccc00700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 0x7f4bb003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.369+0000 7f4bccc00700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 0x7f4bb003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006fd0 tx=0x7f4bb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.374+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4bbc019b90 con 0x7f4bc8107b80 2024-03-16T09:57:45.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.603+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4bc8104360 con 0x7f4bb0038f20 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.604+0000 7f4bc6000700 1 -- 172.21.15.39:0/3868585688 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4bc8104360 con 0x7f4bb0038f20 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.607+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 msgr2=0x7f4bb003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.607+0000 7f4bd0000700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 0x7f4bb003b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4006fd0 tx=0x7f4bb4006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.607+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 msgr2=0x7f4bc819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.607+0000 7f4bd0000700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc011850 tx=0x7f4bbc011c10 comp rx=0 tx=0).stop 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 shutdown_connections 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4bb0038f20 0x7f4bb003b3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 --2- 172.21.15.39:0/3868585688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8107b80 0x7f4bc819de00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 >> 172.21.15.39:0/3868585688 conn(0x7f4bc8103100 msgr2=0x7f4bc8103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 shutdown_connections 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:44.608+0000 7f4bd0000700 1 -- 172.21.15.39:0/3868585688 wait complete. 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:57:45.380 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2024-03-16T09:57:45.381 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2024-03-16T09:57:47.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:47 smithi039 ceph-mon[31510]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:48.431 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:48 smithi039 ceph-mon[31510]: Deploying cephadm binary to smithi039 2024-03-16T09:57:53.591 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2024-03-16T09:57:53.591 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.945+0000 7f0852400700 1 Processor -- start 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.946+0000 7f0852400700 1 -- start start 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.946+0000 7f0852400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.946+0000 7f0852400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f084c07be10 con 0x7f084c106280 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.947+0000 7f0851000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.947+0000 7f0851000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38722/0 (socket says 172.21.15.39:38722) 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.947+0000 7f0851000700 1 -- 172.21.15.39:0/3209026175 learned_addr learned my addr 172.21.15.39:0/3209026175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.947+0000 7f0851000700 1 -- 172.21.15.39:0/3209026175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c106be0 con 0x7f084c106280 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.948+0000 7f0851000700 1 --2- 172.21.15.39:0/3209026175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08340098a0 tx=0x7f0834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcfe26db2f6307e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.948+0000 7f084be00700 1 -- 172.21.15.39:0/3209026175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0834004160 con 0x7f084c106280 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.948+0000 7f084be00700 1 -- 172.21.15.39:0/3209026175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0834004300 con 0x7f084c106280 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.948+0000 7f084be00700 1 -- 172.21.15.39:0/3209026175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0834011b50 con 0x7f084c106280 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 -- 172.21.15.39:0/3209026175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 msgr2=0x7f084c1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 --2- 172.21.15.39:0/3209026175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08340098a0 tx=0x7f0834009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 -- 172.21.15.39:0/3209026175 shutdown_connections 2024-03-16T09:57:53.592 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 --2- 172.21.15.39:0/3209026175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c1066a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 -- 172.21.15.39:0/3209026175 >> 172.21.15.39:0/3209026175 conn(0x7f084c101ea0 msgr2=0x7f084c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 -- 172.21.15.39:0/3209026175 shutdown_connections 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.949+0000 7f0852400700 1 -- 172.21.15.39:0/3209026175 wait complete. 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.950+0000 7f0852400700 1 Processor -- start 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.950+0000 7f0852400700 1 -- start start 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0852400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0852400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f084c107c70 con 0x7f084c106280 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0851000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0851000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38732/0 (socket says 172.21.15.39:38732) 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0851000700 1 -- 172.21.15.39:0/3616322833 learned_addr learned my addr 172.21.15.39:0/3616322833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.951+0000 7f0851000700 1 -- 172.21.15.39:0/3616322833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19e0b0 con 0x7f084c106280 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f0851000700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0834011670 tx=0x7f0834011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08340039c0 con 0x7f084c106280 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0834011e50 con 0x7f084c106280 2024-03-16T09:57:53.593 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f083401a5c0 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19e340 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.952+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a1030 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.953+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c063900 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.954+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f0834024430 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.954+0000 7f084a000700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 0x7f083003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.954+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f083404d2d0 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.954+0000 7f0843e00700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 0x7f083003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.955+0000 7f0843e00700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 0x7f083003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:45.960+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f083404cc20 con 0x7f084c106280 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:46.193+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}) -- 0x7f084c0008d0 con 0x7f0830038f20 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.874+0000 7f084a000700 1 -- 172.21.15.39:0/3616322833 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f084c0008d0 con 0x7f0830038f20 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 msgr2=0x7f083003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:53.594 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 0x7f083003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f083c006fd0 tx=0x7f083c006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 msgr2=0x7f084c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0834011670 tx=0x7f0834011b60 comp rx=0 tx=0).stop 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 shutdown_connections 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f0830038f20 0x7f083003b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 --2- 172.21.15.39:0/3616322833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c106280 0x7f084c19db70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 >> 172.21.15.39:0/3616322833 conn(0x7f084c101ea0 msgr2=0x7f084c1947e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 shutdown_connections 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:52.878+0000 7f0852400700 1 -- 172.21.15.39:0/3616322833 wait complete. 2024-03-16T09:57:53.595 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2024-03-16T09:57:54.213 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:53 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:54.213 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:53 smithi039 ceph-mon[31510]: Added host smithi039 2024-03-16T09:57:54.213 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:53 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.225+0000 7ff888a00700 1 Processor -- start 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.225+0000 7ff888a00700 1 -- start start 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.225+0000 7ff888a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.225+0000 7ff888a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8840823f0 con 0x7ff88410ec90 2024-03-16T09:57:55.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.226+0000 7ff881e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.226+0000 7ff881e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38596/0 (socket says 172.21.15.39:38596) 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.226+0000 7ff881e00700 1 -- 172.21.15.39:0/783219481 learned_addr learned my addr 172.21.15.39:0/783219481 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.226+0000 7ff881e00700 1 -- 172.21.15.39:0/783219481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841115c0 con 0x7ff88410ec90 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.227+0000 7ff881e00700 1 --2- 172.21.15.39:0/783219481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098a0 tx=0x7ff874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a7c63530ed05e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.227+0000 7ff880a00700 1 -- 172.21.15.39:0/783219481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff874004160 con 0x7ff88410ec90 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.227+0000 7ff880a00700 1 -- 172.21.15.39:0/783219481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff874004300 con 0x7ff88410ec90 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.227+0000 7ff880a00700 1 -- 172.21.15.39:0/783219481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff874011b50 con 0x7ff88410ec90 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 -- 172.21.15.39:0/783219481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 msgr2=0x7ff884111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 --2- 172.21.15.39:0/783219481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8740098a0 tx=0x7ff874009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 -- 172.21.15.39:0/783219481 shutdown_connections 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 --2- 172.21.15.39:0/783219481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88410ec90 0x7ff884111080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 -- 172.21.15.39:0/783219481 >> 172.21.15.39:0/783219481 conn(0x7ff88407d1b0 msgr2=0x7ff88407f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 -- 172.21.15.39:0/783219481 shutdown_connections 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.228+0000 7ff888a00700 1 -- 172.21.15.39:0/783219481 wait complete. 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.229+0000 7ff888a00700 1 Processor -- start 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.229+0000 7ff888a00700 1 -- start start 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff888a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff888a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff884112580 con 0x7ff8841a2320 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff881e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff881e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38600/0 (socket says 172.21.15.39:38600) 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff881e00700 1 -- 172.21.15.39:0/3247865213 learned_addr learned my addr 172.21.15.39:0/3247865213 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.230+0000 7ff881e00700 1 -- 172.21.15.39:0/3247865213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841a2c80 con 0x7ff8841a2320 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff881e00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff874011850 tx=0x7ff874011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8740036a0 con 0x7ff8841a2320 2024-03-16T09:57:55.704 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff87401b440 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8841a2f10 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff87401a460 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.231+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a59f0 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.232+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7ff874003840 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.232+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884063900 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.233+0000 7ff87aa00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 0x7ff86803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.233+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff87404d450 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.233+0000 7ff881400700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 0x7ff86803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.234+0000 7ff881400700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 0x7ff86803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.239+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff87401a600 con 0x7ff8841a2320 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.480+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff88407e470 con 0x7ff868038f20 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.486+0000 7ff87aa00700 1 -- 172.21.15.39:0/3247865213 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff88407e470 con 0x7ff868038f20 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 msgr2=0x7ff86803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 0x7ff86803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff870006fd0 tx=0x7ff870006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 msgr2=0x7ff8841a2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff874011850 tx=0x7ff874011c10 comp rx=0 tx=0).stop 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 shutdown_connections 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7ff868038f20 0x7ff86803b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 --2- 172.21.15.39:0/3247865213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8841a2320 0x7ff8841a2740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 >> 172.21.15.39:0/3247865213 conn(0x7ff88407d1b0 msgr2=0x7ff88407dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.489+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 shutdown_connections 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:54.490+0000 7ff888a00700 1 -- 172.21.15.39:0/3247865213 wait complete. 2024-03-16T09:57:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2024-03-16T09:57:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:55 smithi039 ceph-mon[31510]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:55 smithi039 ceph-mon[31510]: Saving service mon spec with placement count:5 2024-03-16T09:57:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:55 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:57.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:57 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:57.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:57 smithi039 ceph-mon[31510]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:57:57.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:57 smithi039 ceph-mon[31510]: Saving service mgr spec with placement count:2 2024-03-16T09:57:57.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:57 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:57.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:57 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:57:57.893 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-16T09:57:57.893 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.642+0000 7f64e5a00700 1 Processor -- start 2024-03-16T09:57:57.893 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.642+0000 7f64e5a00700 1 -- start start 2024-03-16T09:57:57.893 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.642+0000 7f64e5a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:57.893 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.642+0000 7f64e5a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e007bef0 con 0x7f64e0108690 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.643+0000 7f64dea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.643+0000 7f64dea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38608/0 (socket says 172.21.15.39:38608) 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.643+0000 7f64dea00700 1 -- 172.21.15.39:0/3151775022 learned_addr learned my addr 172.21.15.39:0/3151775022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.644+0000 7f64dea00700 1 -- 172.21.15.39:0/3151775022 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e010afc0 con 0x7f64e0108690 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.644+0000 7f64dea00700 1 --2- 172.21.15.39:0/3151775022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64d00098a0 tx=0x7f64d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca96f095ccc16c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.644+0000 7f64dd600700 1 -- 172.21.15.39:0/3151775022 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d0004160 con 0x7f64e0108690 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.644+0000 7f64dd600700 1 -- 172.21.15.39:0/3151775022 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d0004300 con 0x7f64e0108690 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.644+0000 7f64dd600700 1 -- 172.21.15.39:0/3151775022 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d0011b20 con 0x7f64e0108690 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.645+0000 7f64e5a00700 1 -- 172.21.15.39:0/3151775022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 msgr2=0x7f64e010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:57.894 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.645+0000 7f64e5a00700 1 --2- 172.21.15.39:0/3151775022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64d00098a0 tx=0x7f64d0009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.646+0000 7f64e5a00700 1 -- 172.21.15.39:0/3151775022 shutdown_connections 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.646+0000 7f64e5a00700 1 --2- 172.21.15.39:0/3151775022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e010aa80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.646+0000 7f64e5a00700 1 -- 172.21.15.39:0/3151775022 >> 172.21.15.39:0/3151775022 conn(0x7f64e0102000 msgr2=0x7f64e0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.646+0000 7f64e5a00700 1 -- 172.21.15.39:0/3151775022 shutdown_connections 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.646+0000 7f64e5a00700 1 -- 172.21.15.39:0/3151775022 wait complete. 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.647+0000 7f64e5a00700 1 Processor -- start 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.647+0000 7f64e5a00700 1 -- start start 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.648+0000 7f64e5a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.648+0000 7f64e5a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e010c0e0 con 0x7f64e0108690 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.648+0000 7f64dea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.648+0000 7f64dea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38616/0 (socket says 172.21.15.39:38616) 2024-03-16T09:57:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.648+0000 7f64dea00700 1 -- 172.21.15.39:0/3067711998 learned_addr learned my addr 172.21.15.39:0/3067711998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64dea00700 1 -- 172.21.15.39:0/3067711998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e019c170 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64dea00700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64d0011b60 tx=0x7f64d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d0003b60 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d001a910 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64e5a00700 1 -- 172.21.15.39:0/3067711998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e019c400 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d0024930 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.649+0000 7f64e5a00700 1 -- 172.21.15.39:0/3067711998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e019c860 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.651+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f64d0022020 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.651+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b8005330 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.651+0000 7f64d7400700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 0x7f64c403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.651+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f64d004d3b0 con 0x7f64e0108690 2024-03-16T09:57:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.652+0000 7f64de000700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 0x7f64c403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.653+0000 7f64de000700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 0x7f64c403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64cc006fd0 tx=0x7f64cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.657+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f64d0019c70 con 0x7f64e0108690 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.919+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f64b8000bc0 con 0x7f64c4038f70 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.930+0000 7f64d7400700 1 -- 172.21.15.39:0/3067711998 <== mgr.14120 v2:172.21.15.39:6800/1974903299 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f64b8000bc0 con 0x7f64c4038f70 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 msgr2=0x7f64c403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 0x7f64c403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64cc006fd0 tx=0x7f64cc006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 msgr2=0x7f64e019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64d0011b60 tx=0x7f64d0011c40 comp rx=0 tx=0).stop 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 shutdown_connections 2024-03-16T09:57:57.897 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f64c4038f70 0x7f64c403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:57.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 --2- 172.21.15.39:0/3067711998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e0108690 0x7f64e019bc30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:57.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.933+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 >> 172.21.15.39:0/3067711998 conn(0x7f64e0102000 msgr2=0x7f64e0102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:57.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.934+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 shutdown_connections 2024-03-16T09:57:57.898 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:56.934+0000 7f64d4c00700 1 -- 172.21.15.39:0/3067711998 wait complete. 2024-03-16T09:57:58.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:58 smithi039 ceph-mon[31510]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:57:59.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:57:59 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/84780483' entity='client.admin' 2024-03-16T09:57:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.440+0000 7f4d92600700 1 Processor -- start 2024-03-16T09:57:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.440+0000 7f4d92600700 1 -- start start 2024-03-16T09:57:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.440+0000 7f4d92600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.440+0000 7f4d92600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c108580 con 0x7f4d8c107b90 2024-03-16T09:57:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.441+0000 7f4d8b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.441+0000 7f4d8b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38624/0 (socket says 172.21.15.39:38624) 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.441+0000 7f4d8b400700 1 -- 172.21.15.39:0/3712837386 learned_addr learned my addr 172.21.15.39:0/3712837386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.441+0000 7f4d8b400700 1 -- 172.21.15.39:0/3712837386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c108d90 con 0x7f4d8c107b90 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.442+0000 7f4d8b400700 1 --2- 172.21.15.39:0/3712837386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0098a0 tx=0x7f4d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af090aa63916e12 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.442+0000 7f4d8a000700 1 -- 172.21.15.39:0/3712837386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c004160 con 0x7f4d8c107b90 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.442+0000 7f4d8a000700 1 -- 172.21.15.39:0/3712837386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c004300 con 0x7f4d8c107b90 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.442+0000 7f4d8a000700 1 -- 172.21.15.39:0/3712837386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c011b20 con 0x7f4d8c107b90 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 -- 172.21.15.39:0/3712837386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 msgr2=0x7f4d8c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 --2- 172.21.15.39:0/3712837386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0098a0 tx=0x7f4d7c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 -- 172.21.15.39:0/3712837386 shutdown_connections 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 --2- 172.21.15.39:0/3712837386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c107fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 -- 172.21.15.39:0/3712837386 >> 172.21.15.39:0/3712837386 conn(0x7f4d8c103150 msgr2=0x7f4d8c105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 -- 172.21.15.39:0/3712837386 shutdown_connections 2024-03-16T09:57:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.443+0000 7f4d92600700 1 -- 172.21.15.39:0/3712837386 wait complete. 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.444+0000 7f4d92600700 1 Processor -- start 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.444+0000 7f4d92600700 1 -- start start 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d92600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d92600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c10c140 con 0x7f4d8c107b90 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d8b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d8b400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38630/0 (socket says 172.21.15.39:38630) 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d8b400700 1 -- 172.21.15.39:0/84780483 learned_addr learned my addr 172.21.15.39:0/84780483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.445+0000 7f4d8b400700 1 -- 172.21.15.39:0/84780483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c19e230 con 0x7f4d8c107b90 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d8b400700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0117e0 tx=0x7f4d7c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c003bf0 con 0x7f4d8c107b90 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d7c01a910 con 0x7f4d8c107b90 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c024970 con 0x7f4d8c107b90 2024-03-16T09:57:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c19e4c0 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.446+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c19e920 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.447+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8c063900 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.448+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f4d7c022020 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.448+0000 7f4d83e00700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 0x7f4d7003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.448+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d7c04d330 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.449+0000 7f4d8aa00700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 0x7f4d7003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.449+0000 7f4d8aa00700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 0x7f4d7003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d78006fd0 tx=0x7f4d78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.455+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4d7c017ca0 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.689+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4d8c1a16f0 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.692+0000 7f4d83e00700 1 -- 172.21.15.39:0/84780483 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4d7c04cc80 con 0x7f4d8c107b90 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 msgr2=0x7f4d7003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 0x7f4d7003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d78006fd0 tx=0x7f4d78006e40 comp rx=0 tx=0).stop 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 msgr2=0x7f4d8c19dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:57:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0117e0 tx=0x7f4d7c011ba0 comp rx=0 tx=0).stop 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 shutdown_connections 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f4d70038f20 0x7f4d7003b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 --2- 172.21.15.39:0/84780483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d8c107b90 0x7f4d8c19dcf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 >> 172.21.15.39:0/84780483 conn(0x7f4d8c103150 msgr2=0x7f4d8c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.695+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 shutdown_connections 2024-03-16T09:57:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:57:58.696+0000 7f4d92600700 1 -- 172.21.15.39:0/84780483 wait complete. 2024-03-16T09:58:00.797 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:00 smithi039 ceph-mon[31510]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:00.797 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:00 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.549+0000 7fe1c8800700 1 Processor -- start 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.549+0000 7fe1c8800700 1 -- start start 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.549+0000 7fe1c8800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.549+0000 7fe1c8800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c007c870 con 0x7fe1c007de40 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.550+0000 7fe1c5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.550+0000 7fe1c5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58282/0 (socket says 172.21.15.39:58282) 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.550+0000 7fe1c5e00700 1 -- 172.21.15.39:0/3593293319 learned_addr learned my addr 172.21.15.39:0/3593293319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.550+0000 7fe1c5e00700 1 -- 172.21.15.39:0/3593293319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c007c9f0 con 0x7fe1c007de40 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.551+0000 7fe1c5e00700 1 --2- 172.21.15.39:0/3593293319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40098a0 tx=0x7fe1b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39a89751f05f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.552+0000 7fe1c4a00700 1 -- 172.21.15.39:0/3593293319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4004160 con 0x7fe1c007de40 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.552+0000 7fe1c4a00700 1 -- 172.21.15.39:0/3593293319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b4004300 con 0x7fe1c007de40 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.552+0000 7fe1c4a00700 1 -- 172.21.15.39:0/3593293319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4011b20 con 0x7fe1c007de40 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 -- 172.21.15.39:0/3593293319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 msgr2=0x7fe1c007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 --2- 172.21.15.39:0/3593293319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40098a0 tx=0x7fe1b4009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 -- 172.21.15.39:0/3593293319 shutdown_connections 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 --2- 172.21.15.39:0/3593293319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c007c2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 -- 172.21.15.39:0/3593293319 >> 172.21.15.39:0/3593293319 conn(0x7fe1c0103100 msgr2=0x7fe1c0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 -- 172.21.15.39:0/3593293319 shutdown_connections 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.553+0000 7fe1c8800700 1 -- 172.21.15.39:0/3593293319 wait complete. 2024-03-16T09:58:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.555+0000 7fe1c8800700 1 Processor -- start 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.555+0000 7fe1c8800700 1 -- start start 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.555+0000 7fe1c8800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.555+0000 7fe1c8800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c010e200 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.556+0000 7fe1c5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.556+0000 7fe1c5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58288/0 (socket says 172.21.15.39:58288) 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.556+0000 7fe1c5e00700 1 -- 172.21.15.39:0/2194371857 learned_addr learned my addr 172.21.15.39:0/2194371857 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.556+0000 7fe1c5e00700 1 -- 172.21.15.39:0/2194371857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c019e240 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.556+0000 7fe1c5e00700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4011b60 tx=0x7fe1b4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.557+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4003b60 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.557+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b401a910 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.557+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1b4024930 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.557+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c019e4d0 con 0x7fe1c007de40 2024-03-16T09:58:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.557+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c019e930 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.558+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fe1b4022020 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.558+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c0063900 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.559+0000 7fe1baa00700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 0x7fe1a803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.559+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1b404d0a0 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.559+0000 7fe1c5400700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 0x7fe1a803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.560+0000 7fe1c5400700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 0x7fe1a803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.564+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe1b404c9f0 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.795+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe1c01a1390 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.798+0000 7fe1baa00700 1 -- 172.21.15.39:0/2194371857 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe1b400bc70 con 0x7fe1c007de40 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 msgr2=0x7fe1a803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 0x7fe1a803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006fd0 tx=0x7fe1b0006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 msgr2=0x7fe1c019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4011b60 tx=0x7fe1b4011c40 comp rx=0 tx=0).stop 2024-03-16T09:58:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 shutdown_connections 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7fe1a8038f70 0x7fe1a803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 --2- 172.21.15.39:0/2194371857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1c007de40 0x7fe1c019dd00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.802+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 >> 172.21.15.39:0/2194371857 conn(0x7fe1c0103100 msgr2=0x7fe1c0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.803+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 shutdown_connections 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:00.803+0000 7fe1c8800700 1 -- 172.21.15.39:0/2194371857 wait complete. 2024-03-16T09:58:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2024-03-16T09:58:02.017 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:01 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2194371857' entity='client.admin' 2024-03-16T09:58:03.156 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:02 smithi039 ceph-mon[31510]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:03.156 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:02 smithi039 ceph-mon[31510]: from='mgr.14120 172.21.15.39:0/3945929628' entity='mgr.x' 2024-03-16T09:58:03.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:02 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2123111159' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-16T09:58:03.157 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setuser ceph since I am not root 2024-03-16T09:58:03.157 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setgroup ceph since I am not root 2024-03-16T09:58:03.855 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.088+0000 7f1d90800700 1 Processor -- start 2024-03-16T09:58:03.855 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.088+0000 7f1d90800700 1 -- start start 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.089+0000 7f1d90800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.089+0000 7f1d90800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d88108580 con 0x7f1d88107b90 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.089+0000 7f1d8de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.089+0000 7f1d8de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58290/0 (socket says 172.21.15.39:58290) 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.089+0000 7f1d8de00700 1 -- 172.21.15.39:0/3156876708 learned_addr learned my addr 172.21.15.39:0/3156876708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.090+0000 7f1d8de00700 1 -- 172.21.15.39:0/3156876708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d88108d90 con 0x7f1d88107b90 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.090+0000 7f1d8de00700 1 --2- 172.21.15.39:0/3156876708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098a0 tx=0x7f1d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91c53c8a113303c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.090+0000 7f1d8ca00700 1 -- 172.21.15.39:0/3156876708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c004160 con 0x7f1d88107b90 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.090+0000 7f1d8ca00700 1 -- 172.21.15.39:0/3156876708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c004300 con 0x7f1d88107b90 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.090+0000 7f1d8ca00700 1 -- 172.21.15.39:0/3156876708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c011b20 con 0x7f1d88107b90 2024-03-16T09:58:03.856 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.091+0000 7f1d90800700 1 -- 172.21.15.39:0/3156876708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 msgr2=0x7f1d88107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.091+0000 7f1d90800700 1 --2- 172.21.15.39:0/3156876708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098a0 tx=0x7f1d7c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.091+0000 7f1d90800700 1 -- 172.21.15.39:0/3156876708 shutdown_connections 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.091+0000 7f1d90800700 1 --2- 172.21.15.39:0/3156876708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d88107fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.091+0000 7f1d90800700 1 -- 172.21.15.39:0/3156876708 >> 172.21.15.39:0/3156876708 conn(0x7f1d88103150 msgr2=0x7f1d88105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.092+0000 7f1d90800700 1 -- 172.21.15.39:0/3156876708 shutdown_connections 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.092+0000 7f1d90800700 1 -- 172.21.15.39:0/3156876708 wait complete. 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.093+0000 7f1d90800700 1 Processor -- start 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.093+0000 7f1d90800700 1 -- start start 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.093+0000 7f1d90800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.093+0000 7f1d90800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8810c140 con 0x7f1d88107b90 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.094+0000 7f1d8de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.094+0000 7f1d8de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58298/0 (socket says 172.21.15.39:58298) 2024-03-16T09:58:03.860 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.094+0000 7f1d8de00700 1 -- 172.21.15.39:0/2123111159 learned_addr learned my addr 172.21.15.39:0/2123111159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.094+0000 7f1d8de00700 1 -- 172.21.15.39:0/2123111159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8807e070 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.094+0000 7f1d8de00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0117e0 tx=0x7f1d7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.095+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c003bf0 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.095+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c01a910 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.095+0000 7f1d90800700 1 -- 172.21.15.39:0/2123111159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8807c260 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.095+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c0249a0 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.095+0000 7f1d90800700 1 -- 172.21.15.39:0/2123111159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8807c720 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.096+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d88063900 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.097+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1d7c022020 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.097+0000 7f1d82a00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 0x7f1d7003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.097+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1d7c04d130 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.097+0000 7f1d8d400700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 0x7f1d7003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.098+0000 7f1d8d400700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 0x7f1d7003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d78006fd0 tx=0x7f1d78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.107+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1d7c011e20 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.401+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1d881a3640 con 0x7f1d88107b90 2024-03-16T09:58:03.861 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.824+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1d7c00b3e0 con 0x7f1d88107b90 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.828+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 msgr2=0x7f1d7003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.828+0000 7f1d77e00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 0x7f1d7003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d78006fd0 tx=0x7f1d78006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.828+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 msgr2=0x7f1d8807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.828+0000 7f1d82a00700 1 -- 172.21.15.39:0/2123111159 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f1d7c01a430 con 0x7f1d88107b90 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.828+0000 7f1d77e00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0117e0 tx=0x7f1d7c011c10 comp rx=0 tx=0).stop 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.829+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 shutdown_connections 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.829+0000 7f1d77e00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f1d70038f20 0x7f1d7003b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.829+0000 7f1d77e00700 1 --2- 172.21.15.39:0/2123111159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88107b90 0x7f1d8807db30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.829+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 >> 172.21.15.39:0/2123111159 conn(0x7f1d88103150 msgr2=0x7f1d88103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.829+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 shutdown_connections 2024-03-16T09:58:03.862 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:02.830+0000 7f1d77e00700 1 -- 172.21.15.39:0/2123111159 wait complete. 2024-03-16T09:58:04.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2123111159' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-16T09:58:04.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:03 smithi039 ceph-mon[31510]: mgrmap e9: x(active, since 25s) 2024-03-16T09:58:04.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:03.672+0000 7ff9c68b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:58:04.006 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:03.798+0000 7ff9c68b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:58:04.773 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:04.638+0000 7ff9c68b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:58:04.773 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:04.767+0000 7ff9c68b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:58:05.087 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:04.954+0000 7ff9c68b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:58:05.087 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.084+0000 7ff9c68b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:58:05.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3593709920' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T09:58:05.466 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.202+0000 7ff9c68b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:58:05.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.559+0000 7ff9c68b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:58:05.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.704+0000 7ff9c68b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T09:58:05.794 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.804+0000 7f6b17600700 1 Processor -- start 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.804+0000 7f6b17600700 1 -- start start 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.805+0000 7f6b17600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.805+0000 7f6b17600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1007c8b0 con 0x7f6b1007de80 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.805+0000 7f6b14c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.806+0000 7f6b14c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58324/0 (socket says 172.21.15.39:58324) 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.806+0000 7f6b14c00700 1 -- 172.21.15.39:0/3118157071 learned_addr learned my addr 172.21.15.39:0/3118157071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.806+0000 7f6b14c00700 1 -- 172.21.15.39:0/3118157071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1007ca30 con 0x7f6b1007de80 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.806+0000 7f6b14c00700 1 --2- 172.21.15.39:0/3118157071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8416b47eccef3b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.807+0000 7f6b0f400700 1 -- 172.21.15.39:0/3118157071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04004160 con 0x7f6b1007de80 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.807+0000 7f6b0f400700 1 -- 172.21.15.39:0/3118157071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b04004300 con 0x7f6b1007de80 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.807+0000 7f6b0f400700 1 -- 172.21.15.39:0/3118157071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04011b20 con 0x7f6b1007de80 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 -- 172.21.15.39:0/3118157071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 msgr2=0x7f6b1007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 --2- 172.21.15.39:0/3118157071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 -- 172.21.15.39:0/3118157071 shutdown_connections 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 --2- 172.21.15.39:0/3118157071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1007c2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 -- 172.21.15.39:0/3118157071 >> 172.21.15.39:0/3118157071 conn(0x7f6b10103100 msgr2=0x7f6b10105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 -- 172.21.15.39:0/3118157071 shutdown_connections 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.808+0000 7f6b17600700 1 -- 172.21.15.39:0/3118157071 wait complete. 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.809+0000 7f6b17600700 1 Processor -- start 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b17600700 1 -- start start 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b17600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b17600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1010e1c0 con 0x7f6b1007de80 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b14c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b14c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58336/0 (socket says 172.21.15.39:58336) 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.810+0000 7f6b14c00700 1 -- 172.21.15.39:0/3593709920 learned_addr learned my addr 172.21.15.39:0/3593709920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b14c00700 1 -- 172.21.15.39:0/3593709920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1019e250 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b14c00700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b04000c00 tx=0x7f6b04011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04003b60 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0401a910 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b04024930 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.811+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1019e4e0 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.812+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1019e940 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.813+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10063900 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.813+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f6b04022020 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.814+0000 7f6b0d600700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 0x7f6af803b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.814+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b0404d0c0 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.814+0000 7f6b0fe00700 1 -- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 msgr2=0x7f6af803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.814+0000 7f6b0fe00700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 0x7f6af803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:04.819+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b0401ab70 con 0x7f6b1007de80 2024-03-16T09:58:05.797 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.015+0000 7f6b0fe00700 1 -- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 msgr2=0x7f6af803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.015+0000 7f6b0fe00700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 0x7f6af803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.101+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6b101a13e0 con 0x7f6b1007de80 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.102+0000 7f6b0d600700 1 -- 172.21.15.39:0/3593709920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6b0402d3f0 con 0x7f6b1007de80 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 msgr2=0x7f6af803b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 0x7f6af803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 msgr2=0x7f6b1019dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b04000c00 tx=0x7f6b04011c10 comp rx=0 tx=0).stop 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 shutdown_connections 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f6af8038fd0 0x7f6af803b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 --2- 172.21.15.39:0/3593709920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b1007de80 0x7f6b1019dd10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.104+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 >> 172.21.15.39:0/3593709920 conn(0x7f6b10103100 msgr2=0x7f6b10103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.105+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 shutdown_connections 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:05.105+0000 7f6b17600700 1 -- 172.21.15.39:0/3593709920 wait complete. 2024-03-16T09:58:05.798 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-03-16T09:58:05.799 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2024-03-16T09:58:06.096 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.826+0000 7ff9c68b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:58:06.096 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:05.969+0000 7ff9c68b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:58:06.367 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:06.194+0000 7ff9c68b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:58:07.778 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:07.521+0000 7ff9c68b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:58:07.779 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:07.699+0000 7ff9c68b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:58:08.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:08.130+0000 7ff9c68b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:58:08.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:08.253+0000 7ff9c68b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:58:08.781 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:08.523+0000 7ff9c68b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:58:09.170 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:08.779+0000 7ff9c68b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:58:09.496 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:09.167+0000 7ff9c68b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:58:09.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:09.493+0000 7ff9c68b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:58:10.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:09.809+0000 7ff9c68b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:58:11.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:10.892+0000 7ff9c68b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:58:12.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:11.621+0000 7ff9c68b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:58:12.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:11.759+0000 7ff9c68b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:58:12.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:58:12.052+0000 7ff9c68b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:58:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: Active manager daemon x restarted 2024-03-16T09:58:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: Activating manager daemon x 2024-03-16T09:58:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: osdmap e3: 0 total, 0 up, 0 in 2024-03-16T09:58:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: mgrmap e10: x(active, starting, since 0.0100948s) 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:58:12.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:12 smithi039 ceph-mon[31510]: Manager daemon x is now available 2024-03-16T09:58:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:58:13.425 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:58:13.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:13 smithi039 ceph-mon[31510]: mgrmap e11: x(active, since 1.01439s) 2024-03-16T09:58:13.715 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:58:13.715 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-16T09:58:13.715 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T09:58:13.715 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:58:13.716 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.365+0000 7f3677e00700 1 Processor -- start 2024-03-16T09:58:13.716 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.365+0000 7f3677e00700 1 -- start start 2024-03-16T09:58:13.716 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.366+0000 7f3677e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:13.716 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.366+0000 7f3677e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f367807be70 con 0x7f3678106220 2024-03-16T09:58:13.717 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.366+0000 7f3676a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:13.717 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.366+0000 7f3676a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58352/0 (socket says 172.21.15.39:58352) 2024-03-16T09:58:13.717 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.366+0000 7f3676a00700 1 -- 172.21.15.39:0/3304986195 learned_addr learned my addr 172.21.15.39:0/3304986195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.367+0000 7f3676a00700 1 -- 172.21.15.39:0/3304986195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3678106b80 con 0x7f3678106220 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.367+0000 7f3676a00700 1 --2- 172.21.15.39:0/3304986195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36680098a0 tx=0x7f3668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d27cec10c89c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.367+0000 7f3675600700 1 -- 172.21.15.39:0/3304986195 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668004160 con 0x7f3678106220 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.367+0000 7f3675600700 1 -- 172.21.15.39:0/3304986195 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3668004300 con 0x7f3678106220 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.367+0000 7f3675600700 1 -- 172.21.15.39:0/3304986195 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668011b20 con 0x7f3678106220 2024-03-16T09:58:13.718 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 -- 172.21.15.39:0/3304986195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 msgr2=0x7f3678106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 --2- 172.21.15.39:0/3304986195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36680098a0 tx=0x7f3668009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 -- 172.21.15.39:0/3304986195 shutdown_connections 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 --2- 172.21.15.39:0/3304986195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f3678106640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 -- 172.21.15.39:0/3304986195 >> 172.21.15.39:0/3304986195 conn(0x7f3678101ea0 msgr2=0x7f36781042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 -- 172.21.15.39:0/3304986195 shutdown_connections 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.368+0000 7f3677e00700 1 -- 172.21.15.39:0/3304986195 wait complete. 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.369+0000 7f3677e00700 1 Processor -- start 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3677e00700 1 -- start start 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3677e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3677e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3678107c50 con 0x7f3678106220 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3676a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3676a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58356/0 (socket says 172.21.15.39:58356) 2024-03-16T09:58:13.721 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3676a00700 1 -- 172.21.15.39:0/3507757256 learned_addr learned my addr 172.21.15.39:0/3507757256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.370+0000 7f3676a00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367819e0a0 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f3676a00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36680117e0 tx=0x7f3668011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668003ba0 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366801a910 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367819e330 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36680248f0 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.371+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367819e790 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f3668022020 con 0x7f3678106220 2024-03-16T09:58:13.722 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f367ca00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f365c03bb40 con 0x7f365c038f70 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f366804d7e0 con 0x7f3678106220 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f3676000700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.373+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.574+0000 7f3676000700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.574+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.975+0000 7f3676000700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:06.975+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:07.776+0000 7f3676000700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:07.776+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:09.378+0000 7f3676000700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1974903299 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:09.378+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:12.062+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f366802d3f0 con 0x7f3678106220 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:12.062+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 msgr2=0x7f365c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:12.062+0000 7f367ca00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.066+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f366801a430 con 0x7f3678106220 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.066+0000 7f367ca00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 0x7f365c03ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.067+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f365c03bb40 con 0x7f365c03c9f0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.068+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 0x7f365c03ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.069+0000 7f3676000700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 0x7f365c03ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3664003a10 tx=0x7f36640092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.070+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f365c03bb40 con 0x7f365c03c9f0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.077+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3678063900 con 0x7f365c03c9f0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f367ca00700 1 -- 172.21.15.39:0/3507757256 <== mgr.14152 v2:172.21.15.39:6800/1082942412 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3678063900 con 0x7f365c03c9f0 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 msgr2=0x7f365c03ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:13.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 0x7f365c03ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3664003a10 tx=0x7f36640092b0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 msgr2=0x7f367819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36680117e0 tx=0x7f3668011ba0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 shutdown_connections 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3678106220 0x7f367819db60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f365c03c9f0 0x7f365c03ede0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 --2- 172.21.15.39:0/3507757256 >> [v2:172.21.15.39:6800/1974903299,v1:172.21.15.39:6801/1974903299] conn(0x7f365c038f70 0x7f365c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 >> 172.21.15.39:0/3507757256 conn(0x7f3678101ea0 msgr2=0x7f36781947d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.078+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 shutdown_connections 2024-03-16T09:58:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:13.079+0000 7f3677e00700 1 -- 172.21.15.39:0/3507757256 wait complete. 2024-03-16T09:58:13.729 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2024-03-16T09:58:13.729 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2024-03-16T09:58:14.246 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: [16/Mar/2024:09:58:13] ENGINE Bus STARTING 2024-03-16T09:58:14.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T09:58:14.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T09:58:14.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: [16/Mar/2024:09:58:13] ENGINE Serving on https://172.21.15.39:7150 2024-03-16T09:58:14.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: [16/Mar/2024:09:58:13] ENGINE Serving on http://172.21.15.39:8765 2024-03-16T09:58:14.247 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:14 smithi039 ceph-mon[31510]: [16/Mar/2024:09:58:13] ENGINE Bus STARTED 2024-03-16T09:58:15.396 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:15 smithi039 ceph-mon[31510]: mgrmap e12: x(active, since 2s) 2024-03-16T09:58:15.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:15.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:15 smithi039 ceph-mon[31510]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:58:15.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:15.397 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:15.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.584+0000 7f4666000700 1 Processor -- start 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.584+0000 7f4666000700 1 -- start start 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.585+0000 7f4666000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.585+0000 7f4666000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f466010af60 con 0x7f4660108630 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.585+0000 7f465f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.585+0000 7f465f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47054/0 (socket says 172.21.15.39:47054) 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.585+0000 7f465f400700 1 -- 172.21.15.39:0/3051698129 learned_addr learned my addr 172.21.15.39:0/3051698129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.586+0000 7f465f400700 1 -- 172.21.15.39:0/3051698129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466010b0e0 con 0x7f4660108630 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.586+0000 7f465f400700 1 --2- 172.21.15.39:0/3051698129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46500098a0 tx=0x7f4650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4fa5dcdae9ce71 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.586+0000 7f465e000700 1 -- 172.21.15.39:0/3051698129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4650004160 con 0x7f4660108630 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.586+0000 7f465e000700 1 -- 172.21.15.39:0/3051698129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4650004300 con 0x7f4660108630 2024-03-16T09:58:15.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.586+0000 7f465e000700 1 -- 172.21.15.39:0/3051698129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4650011b20 con 0x7f4660108630 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 -- 172.21.15.39:0/3051698129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 msgr2=0x7f466010aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 --2- 172.21.15.39:0/3051698129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46500098a0 tx=0x7f4650009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 -- 172.21.15.39:0/3051698129 shutdown_connections 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 --2- 172.21.15.39:0/3051698129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466010aa20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 -- 172.21.15.39:0/3051698129 >> 172.21.15.39:0/3051698129 conn(0x7f4660102000 msgr2=0x7f4660104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 -- 172.21.15.39:0/3051698129 shutdown_connections 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.587+0000 7f4666000700 1 -- 172.21.15.39:0/3051698129 wait complete. 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.588+0000 7f4666000700 1 Processor -- start 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.588+0000 7f4666000700 1 -- start start 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f4666000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f4666000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f466010c0e0 con 0x7f4660108630 2024-03-16T09:58:15.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f465f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f465f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47062/0 (socket says 172.21.15.39:47062) 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f465f400700 1 -- 172.21.15.39:0/1488993286 learned_addr learned my addr 172.21.15.39:0/1488993286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f465f400700 1 -- 172.21.15.39:0/1488993286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466019e3a0 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.589+0000 7f465f400700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4650011b60 tx=0x7f4650011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.590+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4650003b60 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.590+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465001a910 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.590+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4650024930 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.590+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466019e630 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.590+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466019ea90 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.591+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4660063900 con 0x7f4660108630 2024-03-16T09:58:15.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.591+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f4650022020 con 0x7f4660108630 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.592+0000 7f4657e00700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 0x7f464403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.592+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f465004d1d0 con 0x7f4660108630 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.592+0000 7f465ea00700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 0x7f464403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.593+0000 7f465ea00700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 0x7f464403b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.597+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f465000b3e0 con 0x7f4660108630 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.836+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f46601031e0 con 0x7f4644038fd0 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.973+0000 7f4657e00700 1 -- 172.21.15.39:0/1488993286 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f46601031e0 con 0x7f4644038fd0 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 msgr2=0x7f464403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 0x7f464403b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 msgr2=0x7f466019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4650011b60 tx=0x7f4650011c40 comp rx=0 tx=0).stop 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 shutdown_connections 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4644038fd0 0x7f464403b490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:15.735 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 --2- 172.21.15.39:0/1488993286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4660108630 0x7f466019de60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:15.736 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 >> 172.21.15.39:0/1488993286 conn(0x7f4660102000 msgr2=0x7f4660102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:15.736 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 shutdown_connections 2024-03-16T09:58:15.736 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:14.976+0000 7f4666000700 1 -- 172.21.15.39:0/1488993286 wait complete. 2024-03-16T09:58:15.736 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2024-03-16T09:58:17.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PUMylN3UM/uf8HX3Tjtou.rxMMZ8rgnGxoWcoCy3mqBWbWfFIjjty", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710583096, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-16T09:58:17.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.341+0000 7f5af2800700 1 Processor -- start 2024-03-16T09:58:17.611 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.341+0000 7f5af2800700 1 -- start start 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.341+0000 7f5af2800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.342+0000 7f5af2800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec07c870 con 0x7f5aec07de40 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.342+0000 7f5aeb400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.342+0000 7f5aeb400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47070/0 (socket says 172.21.15.39:47070) 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.342+0000 7f5aeb400700 1 -- 172.21.15.39:0/3112939645 learned_addr learned my addr 172.21.15.39:0/3112939645 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.343+0000 7f5aeb400700 1 -- 172.21.15.39:0/3112939645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec07c9f0 con 0x7f5aec07de40 2024-03-16T09:58:17.612 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.343+0000 7f5aeb400700 1 --2- 172.21.15.39:0/3112939645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0098a0 tx=0x7f5adc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305a4153a9855188 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:17.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.343+0000 7f5aea000700 1 -- 172.21.15.39:0/3112939645 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5adc004160 con 0x7f5aec07de40 2024-03-16T09:58:17.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.343+0000 7f5aea000700 1 -- 172.21.15.39:0/3112939645 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5adc004300 con 0x7f5aec07de40 2024-03-16T09:58:17.615 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.343+0000 7f5aea000700 1 -- 172.21.15.39:0/3112939645 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5adc011b20 con 0x7f5aec07de40 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.344+0000 7f5af2800700 1 -- 172.21.15.39:0/3112939645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 msgr2=0x7f5aec07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.344+0000 7f5af2800700 1 --2- 172.21.15.39:0/3112939645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0098a0 tx=0x7f5adc009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.344+0000 7f5af2800700 1 -- 172.21.15.39:0/3112939645 shutdown_connections 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.344+0000 7f5af2800700 1 --2- 172.21.15.39:0/3112939645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec07c2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.344+0000 7f5af2800700 1 -- 172.21.15.39:0/3112939645 >> 172.21.15.39:0/3112939645 conn(0x7f5aec103100 msgr2=0x7f5aec105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.345+0000 7f5af2800700 1 -- 172.21.15.39:0/3112939645 shutdown_connections 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.345+0000 7f5af2800700 1 -- 172.21.15.39:0/3112939645 wait complete. 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.346+0000 7f5af2800700 1 Processor -- start 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.346+0000 7f5af2800700 1 -- start start 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.346+0000 7f5af2800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.346+0000 7f5af2800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec10e200 con 0x7f5aec07de40 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.346+0000 7f5aeb400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:17.616 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.347+0000 7f5aeb400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47074/0 (socket says 172.21.15.39:47074) 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.347+0000 7f5aeb400700 1 -- 172.21.15.39:0/1981497587 learned_addr learned my addr 172.21.15.39:0/1981497587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.347+0000 7f5aeb400700 1 -- 172.21.15.39:0/1981497587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec19e220 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.347+0000 7f5aeb400700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5adc011b60 tx=0x7f5adc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.348+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5adc003b60 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.348+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5adc01a910 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.348+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec19e4b0 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.348+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5adc024930 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.348+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec19e910 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.349+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f5adc022020 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.349+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec063900 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.350+0000 7f5ae3e00700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 0x7f5ad003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.350+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5adc04d460 con 0x7f5aec07de40 2024-03-16T09:58:17.617 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.350+0000 7f5aeaa00700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 0x7f5ad003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:17.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.351+0000 7f5aeaa00700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 0x7f5ad003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006fd0 tx=0x7f5ad8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:17.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.355+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5adc00bc70 con 0x7f5aec07de40 2024-03-16T09:58:17.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.602+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5aec1a1330 con 0x7f5ad0038fd0 2024-03-16T09:58:17.619 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.885+0000 7f5ae3e00700 1 -- 172.21.15.39:0/1981497587 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5aec1a1330 con 0x7f5ad0038fd0 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.888+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 msgr2=0x7f5ad003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.888+0000 7f5af2800700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 0x7f5ad003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8006fd0 tx=0x7f5ad8006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 msgr2=0x7f5aec19dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5adc011b60 tx=0x7f5adc011c40 comp rx=0 tx=0).stop 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 shutdown_connections 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5ad0038fd0 0x7f5ad003b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 --2- 172.21.15.39:0/1981497587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aec07de40 0x7f5aec19dce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 >> 172.21.15.39:0/1981497587 conn(0x7f5aec103100 msgr2=0x7f5aec103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 shutdown_connections 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:16.889+0000 7f5af2800700 1 -- 172.21.15.39:0/1981497587 wait complete. 2024-03-16T09:58:17.620 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2024-03-16T09:58:18.167 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:17 smithi039 ceph-mon[31510]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:58:18.167 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:17 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:18.167 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:17 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:19.168 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:18 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4277504867' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-16T09:58:19.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2024-03-16T09:58:19.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.165+0000 7fca74200700 1 Processor -- start 2024-03-16T09:58:19.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.165+0000 7fca74200700 1 -- start start 2024-03-16T09:58:19.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.165+0000 7fca74200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.165+0000 7fca74200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c07c8d0 con 0x7fca6c07dea0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.166+0000 7fca71800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.166+0000 7fca71800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47082/0 (socket says 172.21.15.39:47082) 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.166+0000 7fca71800700 1 -- 172.21.15.39:0/1602544338 learned_addr learned my addr 172.21.15.39:0/1602544338 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.166+0000 7fca71800700 1 -- 172.21.15.39:0/1602544338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c07ca50 con 0x7fca6c07dea0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.167+0000 7fca71800700 1 --2- 172.21.15.39:0/1602544338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca600098a0 tx=0x7fca60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cb508f63012b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.167+0000 7fca6be00700 1 -- 172.21.15.39:0/1602544338 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca60004160 con 0x7fca6c07dea0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.167+0000 7fca6be00700 1 -- 172.21.15.39:0/1602544338 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca60004300 con 0x7fca6c07dea0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.167+0000 7fca6be00700 1 -- 172.21.15.39:0/1602544338 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca60011b20 con 0x7fca6c07dea0 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 -- 172.21.15.39:0/1602544338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 msgr2=0x7fca6c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 --2- 172.21.15.39:0/1602544338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca600098a0 tx=0x7fca60009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 -- 172.21.15.39:0/1602544338 shutdown_connections 2024-03-16T09:58:19.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 --2- 172.21.15.39:0/1602544338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c07c300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 -- 172.21.15.39:0/1602544338 >> 172.21.15.39:0/1602544338 conn(0x7fca6c103100 msgr2=0x7fca6c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 -- 172.21.15.39:0/1602544338 shutdown_connections 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.168+0000 7fca74200700 1 -- 172.21.15.39:0/1602544338 wait complete. 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.169+0000 7fca74200700 1 Processor -- start 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca74200700 1 -- start start 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca74200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca74200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c10e1f0 con 0x7fca6c07dea0 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca71800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca71800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47092/0 (socket says 172.21.15.39:47092) 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.170+0000 7fca71800700 1 -- 172.21.15.39:0/4277504867 learned_addr learned my addr 172.21.15.39:0/4277504867 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.171+0000 7fca71800700 1 -- 172.21.15.39:0/4277504867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c19e280 con 0x7fca6c07dea0 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.171+0000 7fca71800700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600116a0 tx=0x7fca60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.171+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca60003a40 con 0x7fca6c07dea0 2024-03-16T09:58:19.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.171+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca60003be0 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.171+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca6001a5e0 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.172+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c19e510 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.172+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c19e970 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.173+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fca600249a0 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.173+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c063900 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.174+0000 7fca6a000700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 0x7fca5403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.174+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fca6004d410 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.174+0000 7fca70e00700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 0x7fca5403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.175+0000 7fca70e00700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 0x7fca5403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca58006fd0 tx=0x7fca58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.179+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca60011e20 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.420+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fca6c1a13e0 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.421+0000 7fca6a000700 1 -- 172.21.15.39:0/4277504867 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fca6002d770 con 0x7fca6c07dea0 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 msgr2=0x7fca5403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:19.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 0x7fca5403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca58006fd0 tx=0x7fca58006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 msgr2=0x7fca6c19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600116a0 tx=0x7fca60011ba0 comp rx=0 tx=0).stop 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 shutdown_connections 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca54038f80 0x7fca5403b440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 --2- 172.21.15.39:0/4277504867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca6c07dea0 0x7fca6c19dd40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.425+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 >> 172.21.15.39:0/4277504867 conn(0x7fca6c103100 msgr2=0x7fca6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.426+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 shutdown_connections 2024-03-16T09:58:19.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:18.426+0000 7fca74200700 1 -- 172.21.15.39:0/4277504867 wait complete. 2024-03-16T09:58:19.935 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:58:19.935 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-03-16T09:58:19.963 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:58:19.963 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-03-16T09:58:19.964 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-03-16T09:58:19.964 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039.front.sepia.ceph.com:8443/ 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout: Password: lmfrsh3qbl 2024-03-16T09:58:19.971 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:19.972 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config directory 2024-03-16T09:58:19.974 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling autotune for osd_memory_target 2024-03-16T09:58:20.133 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:19 smithi039 ceph-mon[31510]: mgrmap e13: x(active, since 6s) 2024-03-16T09:58:21.541 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.644+0000 7efc65000700 1 Processor -- start 2024-03-16T09:58:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.644+0000 7efc65000700 1 -- start start 2024-03-16T09:58:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc65000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc65000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc6007dfd0 con 0x7efc6007c2a0 2024-03-16T09:58:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc5de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc5de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55888/0 (socket says 172.21.15.39:55888) 2024-03-16T09:58:21.543 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc5de00700 1 -- 172.21.15.39:0/2296910401 learned_addr learned my addr 172.21.15.39:0/2296910401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:21.543 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.645+0000 7efc5de00700 1 -- 172.21.15.39:0/2296910401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6007cc00 con 0x7efc6007c2a0 2024-03-16T09:58:21.543 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.646+0000 7efc5de00700 1 --2- 172.21.15.39:0/2296910401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc500098a0 tx=0x7efc50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfe90c7c5aca19e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:21.543 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.646+0000 7efc5ca00700 1 -- 172.21.15.39:0/2296910401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50004160 con 0x7efc6007c2a0 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.646+0000 7efc5ca00700 1 -- 172.21.15.39:0/2296910401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50004300 con 0x7efc6007c2a0 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.646+0000 7efc5ca00700 1 -- 172.21.15.39:0/2296910401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50011b20 con 0x7efc6007c2a0 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 -- 172.21.15.39:0/2296910401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 msgr2=0x7efc6007c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 --2- 172.21.15.39:0/2296910401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efc500098a0 tx=0x7efc50009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 -- 172.21.15.39:0/2296910401 shutdown_connections 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 --2- 172.21.15.39:0/2296910401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6007c6c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:21.544 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 -- 172.21.15.39:0/2296910401 >> 172.21.15.39:0/2296910401 conn(0x7efc601031a0 msgr2=0x7efc601055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 -- 172.21.15.39:0/2296910401 shutdown_connections 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.647+0000 7efc65000700 1 -- 172.21.15.39:0/2296910401 wait complete. 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.648+0000 7efc65000700 1 Processor -- start 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.648+0000 7efc65000700 1 -- start start 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.649+0000 7efc65000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.649+0000 7efc65000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010e2e0 con 0x7efc6007c2a0 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.649+0000 7efc5de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.649+0000 7efc5de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55898/0 (socket says 172.21.15.39:55898) 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.649+0000 7efc5de00700 1 -- 172.21.15.39:0/1193754800 learned_addr learned my addr 172.21.15.39:0/1193754800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc5de00700 1 -- 172.21.15.39:0/1193754800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6019e2e0 con 0x7efc6007c2a0 2024-03-16T09:58:21.547 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc5de00700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc500117e0 tx=0x7efc50011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50003ba0 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5001a910 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6019e570 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.650+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc500248f0 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.651+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6019e9d0 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.652+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7efc50022020 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.652+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc60063900 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.652+0000 7efc56a00700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 0x7efc4403b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.652+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efc5004d590 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.653+0000 7efc5d400700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 0x7efc4403b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.653+0000 7efc5d400700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 0x7efc4403b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.658+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc5001ab70 con 0x7efc6007c2a0 2024-03-16T09:58:21.548 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.881+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7efc601a1260 con 0x7efc6007c2a0 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.882+0000 7efc56a00700 1 -- 172.21.15.39:0/1193754800 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7efc601a1260 con 0x7efc6007c2a0 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 msgr2=0x7efc4403b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 0x7efc4403b530 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc4c006fd0 tx=0x7efc4c006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 msgr2=0x7efc6019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc500117e0 tx=0x7efc50011ba0 comp rx=0 tx=0).stop 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 shutdown_connections 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc44039070 0x7efc4403b530 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 --2- 172.21.15.39:0/1193754800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc6007c2a0 0x7efc6019dda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 >> 172.21.15.39:0/1193754800 conn(0x7efc601031a0 msgr2=0x7efc60103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 shutdown_connections 2024-03-16T09:58:21.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:20.885+0000 7efc65000700 1 -- 172.21.15.39:0/1193754800 wait complete. 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.206+0000 7f824e200700 1 Processor -- start 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.206+0000 7f824e200700 1 -- start start 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.206+0000 7f824e200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.206+0000 7f824e200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f824807c850 con 0x7f824807de20 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.207+0000 7f8247400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.207+0000 7f8247400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55904/0 (socket says 172.21.15.39:55904) 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.207+0000 7f8247400700 1 -- 172.21.15.39:0/1399329279 learned_addr learned my addr 172.21.15.39:0/1399329279 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:23.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.207+0000 7f8247400700 1 -- 172.21.15.39:0/1399329279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824807c9d0 con 0x7f824807de20 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.208+0000 7f8247400700 1 --2- 172.21.15.39:0/1399329279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82380098a0 tx=0x7f8238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c6f86252a19eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.208+0000 7f8246a00700 1 -- 172.21.15.39:0/1399329279 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8238004160 con 0x7f824807de20 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.208+0000 7f8246a00700 1 -- 172.21.15.39:0/1399329279 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8238004300 con 0x7f824807de20 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.208+0000 7f8246a00700 1 -- 172.21.15.39:0/1399329279 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8238011b20 con 0x7f824807de20 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 -- 172.21.15.39:0/1399329279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 msgr2=0x7f824807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 --2- 172.21.15.39:0/1399329279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82380098a0 tx=0x7f8238009bb0 comp rx=0 tx=0).stop 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 -- 172.21.15.39:0/1399329279 shutdown_connections 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 --2- 172.21.15.39:0/1399329279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824807c280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 -- 172.21.15.39:0/1399329279 >> 172.21.15.39:0/1399329279 conn(0x7f8248103100 msgr2=0x7f8248105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 -- 172.21.15.39:0/1399329279 shutdown_connections 2024-03-16T09:58:23.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.209+0000 7f824e200700 1 -- 172.21.15.39:0/1399329279 wait complete. 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.210+0000 7f824e200700 1 Processor -- start 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.210+0000 7f824e200700 1 -- start start 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f824e200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f824e200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f824810e1c0 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f8247400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f8247400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55908/0 (socket says 172.21.15.39:55908) 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f8247400700 1 -- 172.21.15.39:0/3015238844 learned_addr learned my addr 172.21.15.39:0/3015238844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.211+0000 7f8247400700 1 -- 172.21.15.39:0/3015238844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824819e260 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f8247400700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82380116a0 tx=0x7f8238011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8238003a40 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8238003be0 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823801a750 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824819e4f0 con 0x7f824807de20 2024-03-16T09:58:23.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.212+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824819e950 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.214+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8248063900 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.214+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8238024aa0 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.214+0000 7f8244c00700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 0x7f822c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.214+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f823804d6b0 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.215+0000 7f823ea00700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 0x7f822c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.215+0000 7f823ea00700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 0x7f822c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.220+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8238011e20 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.565+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f82481a13b0 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.571+0000 7f8244c00700 1 -- 172.21.15.39:0/3015238844 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f823802d770 con 0x7f824807de20 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 msgr2=0x7f822c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 0x7f822c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 msgr2=0x7f824819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82380116a0 tx=0x7f8238011ba0 comp rx=0 tx=0).stop 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 shutdown_connections 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f822c039020 0x7f822c03b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:23.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 --2- 172.21.15.39:0/3015238844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824807de20 0x7f824819dd20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:23.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 >> 172.21.15.39:0/3015238844 conn(0x7f8248103100 msgr2=0x7f8248103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:23.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 shutdown_connections 2024-03-16T09:58:23.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2024-03-16T09:58:22.574+0000 7f824e200700 1 -- 172.21.15.39:0/3015238844 wait complete. 2024-03-16T09:58:23.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-16T09:58:23.200 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:23.201 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2024-03-16T09:58:23.324 INFO:tasks.cephadm:Fetching config... 2024-03-16T09:58:23.324 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:58:23.324 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-16T09:58:23.377 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-16T09:58:23.377 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:58:23.377 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-16T09:58:23.417 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-16T09:58:23.418 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:58:23.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-16T09:58:23.550 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-16T09:58:23.551 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:58:23.551 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-16T09:58:23.598 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-16T09:58:23.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:58:23.766 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:58:23.767 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:23 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3015238844' entity='client.admin' 2024-03-16T09:58:23.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:58:23.867 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:58:23.882 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:58:23.927 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs6SjoqGQG1FbFuv2uoGLUwhf1mgAlMIor4Z1H/4V2OpzPN7jXNhCq+1XEAIBw1TLjj0OT6fvC9kI4zbSpNOCPyu2b7nanLYBXtgNSmHzlyetNGD5kW5iz12yCyhpL1O5QdzmplAmUYjluwcUs3UVmUDXlmBl67fkdyDvAKr8CSGmVDUMTXLdTI2xGnJf82npluVcCWLhwMA/QUlucesNXK5CMCojNmC8I/ExqbOrQoJK5rmy63rBefDlUBvx7AvxkojN8N2zSLjyBnmX2HsdADgk0Vl8/1y7fmWeeiQzQOogB5VZweH1wgzYSjFOgzrYX5wo21rY4eVSNSMxwMqMLjJSInEWQBzU9zdfB5JvqN10o8eAx3fvbt7GMkRePXyU3pdLiqDzpBk2KcmrZvMm1ofPaQO+bIWbnJHx4LmHOvRezNLptN9LBuCCioXj2Jeshc+C9Ii0cufrTYFMwOIvYjEMZy8lxToQNTeYSYpOSvAgPKixOzmOprRVF74K4XM8= ceph-65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:58:23.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-16T09:58:24.378 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 -- 172.21.15.39:0/2229432004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 msgr2=0x7fb96c101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 --2- 172.21.15.39:0/2229432004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c101f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009910 tx=0x7fb95c009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 -- 172.21.15.39:0/2229432004 shutdown_connections 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 --2- 172.21.15.39:0/2229432004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c101f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 -- 172.21.15.39:0/2229432004 >> 172.21.15.39:0/2229432004 conn(0x7fb96c0fd0e0 msgr2=0x7fb96c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 -- 172.21.15.39:0/2229432004 shutdown_connections 2024-03-16T09:58:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.002+0000 7fb96b200700 1 -- 172.21.15.39:0/2229432004 wait complete. 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.003+0000 7fb96b200700 1 Processor -- start 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb96b200700 1 -- start start 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb96b200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb96b200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c107bf0 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb969e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb969e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55924/0 (socket says 172.21.15.39:55924) 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.004+0000 7fb969e00700 1 -- 172.21.15.39:0/248816252 learned_addr learned my addr 172.21.15.39:0/248816252 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb969e00700 1 -- 172.21.15.39:0/248816252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c197780 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb969e00700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c027070 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c00be40 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c00f460 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c197a10 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.005+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c19a700 con 0x7fb96c101b60 2024-03-16T09:58:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.007+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fb95c02a070 con 0x7fb96c101b60 2024-03-16T09:58:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.007+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c108370 con 0x7fb96c101b60 2024-03-16T09:58:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.007+0000 7fb962a00700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 0x7fb95003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.007+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb95c01e030 con 0x7fb96c101b60 2024-03-16T09:58:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.008+0000 7fb963e00700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 0x7fb95003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.008+0000 7fb963e00700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 0x7fb95003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb954006fd0 tx=0x7fb954006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.014+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb95c013070 con 0x7fb96c101b60 2024-03-16T09:58:27.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.243+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb96c02dc80 con 0x7fb96c101b60 2024-03-16T09:58:27.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.249+0000 7fb962a00700 1 -- 172.21.15.39:0/248816252 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb95c031020 con 0x7fb96c101b60 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.252+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 msgr2=0x7fb95003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.252+0000 7fb96b200700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 0x7fb95003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb954006fd0 tx=0x7fb954006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.252+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 msgr2=0x7fb96c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.252+0000 7fb96b200700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb95c006010 tx=0x7fb95c005050 comp rx=0 tx=0).stop 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 shutdown_connections 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb950038fc0 0x7fb95003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 --2- 172.21.15.39:0/248816252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb96c101b60 0x7fb96c197240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 >> 172.21.15.39:0/248816252 conn(0x7fb96c0fd0e0 msgr2=0x7fb96c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:27.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 shutdown_connections 2024-03-16T09:58:27.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:27.253+0000 7fb96b200700 1 -- 172.21.15.39:0/248816252 wait complete. 2024-03-16T09:58:28.038 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-16T09:58:28.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-16T09:58:28.456 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:28.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:28 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/248816252' entity='client.admin' 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:31.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.038+0000 7f03eca00700 1 -- 172.21.15.39:0/701312421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 msgr2=0x7f03e8103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:31.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.038+0000 7f03eca00700 1 --2- 172.21.15.39:0/701312421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8103800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009910 tx=0x7f03d8009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:31.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.038+0000 7f03eca00700 1 -- 172.21.15.39:0/701312421 shutdown_connections 2024-03-16T09:58:31.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.038+0000 7f03eca00700 1 --2- 172.21.15.39:0/701312421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8103800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:31.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.038+0000 7f03eca00700 1 -- 172.21.15.39:0/701312421 >> 172.21.15.39:0/701312421 conn(0x7f03e80fd2c0 msgr2=0x7f03e80ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:31.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.039+0000 7f03eca00700 1 -- 172.21.15.39:0/701312421 shutdown_connections 2024-03-16T09:58:31.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.039+0000 7f03eca00700 1 -- 172.21.15.39:0/701312421 wait complete. 2024-03-16T09:58:31.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.040+0000 7f03eca00700 1 Processor -- start 2024-03-16T09:58:31.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.040+0000 7f03eca00700 1 -- start start 2024-03-16T09:58:31.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.040+0000 7f03eca00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:31.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.040+0000 7f03eca00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8107ea0 con 0x7f03e81033e0 2024-03-16T09:58:31.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.040+0000 7f03e5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:31.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.041+0000 7f03e5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49120/0 (socket says 172.21.15.39:49120) 2024-03-16T09:58:31.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.041+0000 7f03e5e00700 1 -- 172.21.15.39:0/61495109 learned_addr learned my addr 172.21.15.39:0/61495109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:31.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.041+0000 7f03e5e00700 1 -- 172.21.15.39:0/61495109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8197a60 con 0x7f03e81033e0 2024-03-16T09:58:31.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.041+0000 7f03e5e00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005850 tx=0x7f03d8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:31.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.041+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d8026070 con 0x7f03e81033e0 2024-03-16T09:58:31.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.042+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d80044a0 con 0x7f03e81033e0 2024-03-16T09:58:31.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.042+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e8197cf0 con 0x7f03e81033e0 2024-03-16T09:58:31.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.042+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03d8003ab0 con 0x7f03e81033e0 2024-03-16T09:58:31.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.042+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e819a9e0 con 0x7f03e81033e0 2024-03-16T09:58:31.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.043+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e8108620 con 0x7f03e81033e0 2024-03-16T09:58:31.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.043+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f03d8029070 con 0x7f03e81033e0 2024-03-16T09:58:31.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.044+0000 7f03dea00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 0x7f03cc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:31.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.044+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f03d801e070 con 0x7f03e81033e0 2024-03-16T09:58:31.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.044+0000 7f03e5400700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 0x7f03cc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:31.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.045+0000 7f03e5400700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 0x7f03cc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006fd0 tx=0x7f03d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:31.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.049+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03d8013070 con 0x7f03e81033e0 2024-03-16T09:58:31.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.283+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f03e8062710 con 0x7f03cc039020 2024-03-16T09:58:31.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.291+0000 7f03dea00700 1 -- 172.21.15.39:0/61495109 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f03e8062710 con 0x7f03cc039020 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 msgr2=0x7f03cc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 0x7f03cc03b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006fd0 tx=0x7f03d4006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 msgr2=0x7f03e8197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005850 tx=0x7f03d8005030 comp rx=0 tx=0).stop 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 shutdown_connections 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03cc039020 0x7f03cc03b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:31.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 --2- 172.21.15.39:0/61495109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e81033e0 0x7f03e8197520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:31.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 >> 172.21.15.39:0/61495109 conn(0x7f03e80fd2c0 msgr2=0x7f03e80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:31.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.294+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 shutdown_connections 2024-03-16T09:58:31.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:31.295+0000 7f03eca00700 1 -- 172.21.15.39:0/61495109 wait complete. 2024-03-16T09:58:32.070 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi105 2024-03-16T09:58:32.071 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:58:32.071 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.conf 2024-03-16T09:58:32.090 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:58:32.090 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:32.147 INFO:tasks.cephadm:Adding host smithi105 to orchestrator... 2024-03-16T09:58:32.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch host add smithi105 2024-03-16T09:58:32.589 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:32.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:58:32.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:32.734 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:32.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:58:32.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:58:32.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T09:58:32.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:32 smithi039 ceph-mon[31510]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:58:33.727 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:33 smithi039 ceph-mon[31510]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:33.728 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:33 smithi039 ceph-mon[31510]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 -- 172.21.15.39:0/1285357609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 msgr2=0x7f8dac104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 --2- 172.21.15.39:0/1285357609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac104360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009910 tx=0x7f8da0009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 -- 172.21.15.39:0/1285357609 shutdown_connections 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 --2- 172.21.15.39:0/1285357609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac104360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 -- 172.21.15.39:0/1285357609 >> 172.21.15.39:0/1285357609 conn(0x7f8dac0ff810 msgr2=0x7f8dac101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 -- 172.21.15.39:0/1285357609 shutdown_connections 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.497+0000 7f8db2200700 1 -- 172.21.15.39:0/1285357609 wait complete. 2024-03-16T09:58:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.499+0000 7f8db2200700 1 Processor -- start 2024-03-16T09:58:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.499+0000 7f8db2200700 1 -- start start 2024-03-16T09:58:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.499+0000 7f8db2200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.500+0000 7f8db2200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac10e350 con 0x7f8dac103f80 2024-03-16T09:58:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.500+0000 7f8db0e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.500+0000 7f8db0e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49142/0 (socket says 172.21.15.39:49142) 2024-03-16T09:58:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.500+0000 7f8db0e00700 1 -- 172.21.15.39:0/3711344211 learned_addr learned my addr 172.21.15.39:0/3711344211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.500+0000 7f8db0e00700 1 -- 172.21.15.39:0/3711344211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac19fab0 con 0x7f8dac103f80 2024-03-16T09:58:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8db0e00700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005850 tx=0x7f8da0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:34.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8da0026070 con 0x7f8dac103f80 2024-03-16T09:58:34.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da000bbd0 con 0x7f8dac103f80 2024-03-16T09:58:34.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac19fda0 con 0x7f8dac103f80 2024-03-16T09:58:34.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8da0003970 con 0x7f8dac103f80 2024-03-16T09:58:34.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.501+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac199810 con 0x7f8dac103f80 2024-03-16T09:58:34.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.502+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f8da0029070 con 0x7f8dac103f80 2024-03-16T09:58:34.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.502+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dac105460 con 0x7f8dac103f80 2024-03-16T09:58:34.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.503+0000 7f8da9600700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 0x7f8d9403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:34.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.503+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8da001e070 con 0x7f8dac103f80 2024-03-16T09:58:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.503+0000 7f8dabe00700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 0x7f8d9403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.504+0000 7f8dabe00700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 0x7f8d9403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006fd0 tx=0x7f8d98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:34.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.509+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8da0013070 con 0x7f8dac103f80 2024-03-16T09:58:34.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:34.739+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}) -- 0x7f8dac19aba0 con 0x7f8d94038fc0 2024-03-16T09:58:35.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:34 smithi039 ceph-mon[31510]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:58:35.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:35.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:35.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:36.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:35 smithi039 ceph-mon[31510]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:36.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:35 smithi039 ceph-mon[31510]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:58:37.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:36 smithi039 ceph-mon[31510]: Deploying cephadm binary to smithi105 2024-03-16T09:58:38.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:37 smithi039 ceph-mon[31510]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.622+0000 7f8da9600700 1 -- 172.21.15.39:0/3711344211 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8dac19aba0 con 0x7f8d94038fc0 2024-03-16T09:58:39.623 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi105' with addr '172.21.15.105' 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 msgr2=0x7f8d9403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 0x7f8d9403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006fd0 tx=0x7f8d98006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 msgr2=0x7f8dac19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8da0005850 tx=0x7f8da0005030 comp rx=0 tx=0).stop 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 shutdown_connections 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8d94038fc0 0x7f8d9403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 --2- 172.21.15.39:0/3711344211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dac103f80 0x7f8dac19f570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 >> 172.21.15.39:0/3711344211 conn(0x7f8dac0ff810 msgr2=0x7f8dac105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 shutdown_connections 2024-03-16T09:58:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:39.626+0000 7f8db2200700 1 -- 172.21.15.39:0/3711344211 wait complete. 2024-03-16T09:58:40.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:39 smithi039 ceph-mon[31510]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:40.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:40.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:40.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T09:58:40.855 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:41.226 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:40 smithi039 ceph-mon[31510]: Added host smithi105 2024-03-16T09:58:41.226 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:41.226 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:42.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:41 smithi039 ceph-mon[31510]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 -- 172.21.15.39:0/306080944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 msgr2=0x7f57d8103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 --2- 172.21.15.39:0/306080944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8103240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009910 tx=0x7f57c8009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 -- 172.21.15.39:0/306080944 shutdown_connections 2024-03-16T09:58:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 --2- 172.21.15.39:0/306080944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8103240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 -- 172.21.15.39:0/306080944 >> 172.21.15.39:0/306080944 conn(0x7f57d8077810 msgr2=0x7f57d80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 -- 172.21.15.39:0/306080944 shutdown_connections 2024-03-16T09:58:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.795+0000 7f57dd400700 1 -- 172.21.15.39:0/306080944 wait complete. 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.796+0000 7f57dd400700 1 Processor -- start 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.796+0000 7f57dd400700 1 -- start start 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.797+0000 7f57dd400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.797+0000 7f57dd400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57d810c350 con 0x7f57d8102e60 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.797+0000 7f57d6800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.797+0000 7f57d6800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52920/0 (socket says 172.21.15.39:52920) 2024-03-16T09:58:42.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.797+0000 7f57d6800700 1 -- 172.21.15.39:0/2500709782 learned_addr learned my addr 172.21.15.39:0/2500709782 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57d6800700 1 -- 172.21.15.39:0/2500709782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d819a020 con 0x7f57d8102e60 2024-03-16T09:58:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57d6800700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57c8006010 tx=0x7f57c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c8026070 con 0x7f57d8102e60 2024-03-16T09:58:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c800bc30 con 0x7f57d8102e60 2024-03-16T09:58:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d819a2b0 con 0x7f57d8102e60 2024-03-16T09:58:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c80039a0 con 0x7f57d8102e60 2024-03-16T09:58:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.798+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d819a710 con 0x7f57d8102e60 2024-03-16T09:58:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.799+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8069fa0 con 0x7f57d8102e60 2024-03-16T09:58:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.800+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f57c8029070 con 0x7f57d8102e60 2024-03-16T09:58:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.800+0000 7f57cf400700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 0x7f57bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.800+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f57c801e070 con 0x7f57d8102e60 2024-03-16T09:58:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.801+0000 7f57d5e00700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 0x7f57bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.801+0000 7f57d5e00700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 0x7f57bc03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006fd0 tx=0x7f57c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:42.806+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57c8013070 con 0x7f57d8102e60 2024-03-16T09:58:43.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.037+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57d8108730 con 0x7f57bc038fc0 2024-03-16T09:58:43.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.039+0000 7f57cf400700 1 -- 172.21.15.39:0/2500709782 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f57d8108730 con 0x7f57bc038fc0 2024-03-16T09:58:43.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:43.041 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}] 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 msgr2=0x7f57bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 0x7f57bc03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f57c4006fd0 tx=0x7f57c4006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 msgr2=0x7f57d8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f57c8006010 tx=0x7f57c8005030 comp rx=0 tx=0).stop 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 shutdown_connections 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f57bc038fc0 0x7f57bc03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 --2- 172.21.15.39:0/2500709782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57d8102e60 0x7f57d8199ae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 >> 172.21.15.39:0/2500709782 conn(0x7f57d8077810 msgr2=0x7f57d8108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 shutdown_connections 2024-03-16T09:58:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:43.042+0000 7f57dd400700 1 -- 172.21.15.39:0/2500709782 wait complete. 2024-03-16T09:58:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:43 smithi039 ceph-mon[31510]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:43.826 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-03-16T09:58:43.826 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:58:43.826 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-03-16T09:58:43.846 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:58:43.846 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:43.903 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-03-16T09:58:43.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch host add smithi194 2024-03-16T09:58:44.363 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:44.630 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:44 smithi039 ceph-mon[31510]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:58:45.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:45 smithi039 ceph-mon[31510]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:45.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:45.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:45.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:45.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.413+0000 7f5f53e00700 1 -- 172.21.15.39:0/4047113926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 msgr2=0x7f5f54101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.413+0000 7f5f53e00700 1 --2- 172.21.15.39:0/4047113926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f54101920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009910 tx=0x7f5f44009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.414+0000 7f5f53e00700 1 -- 172.21.15.39:0/4047113926 shutdown_connections 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.414+0000 7f5f53e00700 1 --2- 172.21.15.39:0/4047113926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f54101920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.414+0000 7f5f53e00700 1 -- 172.21.15.39:0/4047113926 >> 172.21.15.39:0/4047113926 conn(0x7f5f54076560 msgr2=0x7f5f54076970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.414+0000 7f5f53e00700 1 -- 172.21.15.39:0/4047113926 shutdown_connections 2024-03-16T09:58:46.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.414+0000 7f5f53e00700 1 -- 172.21.15.39:0/4047113926 wait complete. 2024-03-16T09:58:46.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.416+0000 7f5f53e00700 1 Processor -- start 2024-03-16T09:58:46.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.416+0000 7f5f53e00700 1 -- start start 2024-03-16T09:58:46.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.417+0000 7f5f53e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.417+0000 7f5f53e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5410e3b0 con 0x7f5f54101540 2024-03-16T09:58:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.417+0000 7f5f52a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.418+0000 7f5f52a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52942/0 (socket says 172.21.15.39:52942) 2024-03-16T09:58:46.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.418+0000 7f5f52a00700 1 -- 172.21.15.39:0/1462771443 learned_addr learned my addr 172.21.15.39:0/1462771443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.418+0000 7f5f52a00700 1 -- 172.21.15.39:0/1462771443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5419fb30 con 0x7f5f54101540 2024-03-16T09:58:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.418+0000 7f5f52a00700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005850 tx=0x7f5f44005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.419+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f44026070 con 0x7f5f54101540 2024-03-16T09:58:46.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.419+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5419fdc0 con 0x7f5f54101540 2024-03-16T09:58:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.419+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f4400bc30 con 0x7f5f54101540 2024-03-16T09:58:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.419+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f44003a20 con 0x7f5f54101540 2024-03-16T09:58:46.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.419+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f541996c0 con 0x7f5f54101540 2024-03-16T09:58:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.420+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f5f44029070 con 0x7f5f54101540 2024-03-16T09:58:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.420+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f54101050 con 0x7f5f54101540 2024-03-16T09:58:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.420+0000 7f5f4b400700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 0x7f5f3803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.420+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f4401e070 con 0x7f5f54101540 2024-03-16T09:58:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.421+0000 7f5f52000700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 0x7f5f3803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.421+0000 7f5f52000700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 0x7f5f3803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.426+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f44013070 con 0x7f5f54101540 2024-03-16T09:58:46.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:58:46.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:58:46.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:58:46.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T09:58:46.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:58:46.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:46 smithi039 ceph-mon[31510]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:46.664+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) -- 0x7f5f5419abf0 con 0x7f5f38038fc0 2024-03-16T09:58:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:48.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:58:48.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:48.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:48.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:48.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:47 smithi039 ceph-mon[31510]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:58:49.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:48 smithi039 ceph-mon[31510]: Deploying cephadm binary to smithi194 2024-03-16T09:58:50.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:49 smithi039 ceph-mon[31510]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:51.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.496+0000 7f5f4b400700 1 -- 172.21.15.39:0/1462771443 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5f5419abf0 con 0x7f5f38038fc0 2024-03-16T09:58:51.498 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-03-16T09:58:51.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 msgr2=0x7f5f3803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:51.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 0x7f5f3803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:51.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 msgr2=0x7f5f5419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:51.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f44005850 tx=0x7f5f44005030 comp rx=0 tx=0).stop 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 shutdown_connections 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5f38038fc0 0x7f5f3803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 --2- 172.21.15.39:0/1462771443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f54101540 0x7f5f5419f5f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 >> 172.21.15.39:0/1462771443 conn(0x7f5f54076560 msgr2=0x7f5f54109030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.501+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 shutdown_connections 2024-03-16T09:58:51.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:51.502+0000 7f5f53e00700 1 -- 172.21.15.39:0/1462771443 wait complete. 2024-03-16T09:58:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:51 smithi039 ceph-mon[31510]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:58:52.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T09:58:52.709 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:52 smithi039 ceph-mon[31510]: Added host smithi194 2024-03-16T09:58:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:52 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:52 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:53.767 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:53 smithi039 ceph-mon[31510]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.532+0000 7f0a93600700 1 -- 172.21.15.39:0/226245500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 msgr2=0x7f0a8c104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.532+0000 7f0a93600700 1 --2- 172.21.15.39:0/226245500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c104510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009910 tx=0x7f0a80009c20 comp rx=0 tx=0).stop 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.532+0000 7f0a93600700 1 -- 172.21.15.39:0/226245500 shutdown_connections 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.532+0000 7f0a93600700 1 --2- 172.21.15.39:0/226245500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c104510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.532+0000 7f0a93600700 1 -- 172.21.15.39:0/226245500 >> 172.21.15.39:0/226245500 conn(0x7f0a8c0ff9c0 msgr2=0x7f0a8c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.533+0000 7f0a93600700 1 -- 172.21.15.39:0/226245500 shutdown_connections 2024-03-16T09:58:54.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.533+0000 7f0a93600700 1 -- 172.21.15.39:0/226245500 wait complete. 2024-03-16T09:58:54.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.534+0000 7f0a93600700 1 Processor -- start 2024-03-16T09:58:54.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.534+0000 7f0a93600700 1 -- start start 2024-03-16T09:58:54.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.535+0000 7f0a93600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:54.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.535+0000 7f0a93600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c10e500 con 0x7f0a8c104130 2024-03-16T09:58:54.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.535+0000 7f0a90c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:54.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.535+0000 7f0a90c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41898/0 (socket says 172.21.15.39:41898) 2024-03-16T09:58:54.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.535+0000 7f0a90c00700 1 -- 172.21.15.39:0/214334339 learned_addr learned my addr 172.21.15.39:0/214334339 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:54.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a90c00700 1 -- 172.21.15.39:0/214334339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19fbb0 con 0x7f0a8c104130 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a90c00700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005850 tx=0x7f0a80005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80026070 con 0x7f0a8c104130 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a8000bbd0 con 0x7f0a8c104130 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19fea0 con 0x7f0a8c104130 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a80003970 con 0x7f0a8c104130 2024-03-16T09:58:54.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.536+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1997f0 con 0x7f0a8c104130 2024-03-16T09:58:54.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.537+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f0a80029070 con 0x7f0a8c104130 2024-03-16T09:58:54.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.537+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c105610 con 0x7f0a8c104130 2024-03-16T09:58:54.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.538+0000 7f0a89600700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 0x7f0a7403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:54.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.538+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a8001e070 con 0x7f0a8c104130 2024-03-16T09:58:54.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.538+0000 7f0a8be00700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 0x7f0a7403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:54.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.539+0000 7f0a8be00700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 0x7f0a7403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.544+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0a80013070 con 0x7f0a8c104130 2024-03-16T09:58:54.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.774+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a8c19ad20 con 0x7f0a74039020 2024-03-16T09:58:54.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.777+0000 7f0a89600700 1 -- 172.21.15.39:0/214334339 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f0a8c19ad20 con 0x7f0a74039020 2024-03-16T09:58:54.779 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T09:58:54.779 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 msgr2=0x7f0a7403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 0x7f0a7403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 msgr2=0x7f0a8c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a80005850 tx=0x7f0a80005030 comp rx=0 tx=0).stop 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 shutdown_connections 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0a74039020 0x7f0a7403b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 --2- 172.21.15.39:0/214334339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a8c104130 0x7f0a8c19f670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 >> 172.21.15.39:0/214334339 conn(0x7f0a8c0ff9c0 msgr2=0x7f0a8c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:54.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 shutdown_connections 2024-03-16T09:58:54.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:54.780+0000 7f0a93600700 1 -- 172.21.15.39:0/214334339 wait complete. 2024-03-16T09:58:55.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:55 smithi039 ceph-mon[31510]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:55.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:55.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:55 smithi039 ceph-mon[31510]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:58:55.491 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-16T09:58:55.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-16T09:58:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:58:57.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:57 smithi039 ceph-mon[31510]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:57.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.973+0000 7ff02e200700 1 -- 172.21.15.39:0/1359976792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 msgr2=0x7ff028103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.973+0000 7ff02e200700 1 --2- 172.21.15.39:0/1359976792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028103210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff01800b0a0 tx=0x7ff01800b3b0 comp rx=0 tx=0).stop 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.974+0000 7ff02e200700 1 -- 172.21.15.39:0/1359976792 shutdown_connections 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.974+0000 7ff02e200700 1 --2- 172.21.15.39:0/1359976792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028103210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.974+0000 7ff02e200700 1 -- 172.21.15.39:0/1359976792 >> 172.21.15.39:0/1359976792 conn(0x7ff0280777d0 msgr2=0x7ff0280feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.974+0000 7ff02e200700 1 -- 172.21.15.39:0/1359976792 shutdown_connections 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.974+0000 7ff02e200700 1 -- 172.21.15.39:0/1359976792 wait complete. 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.975+0000 7ff02e200700 1 Processor -- start 2024-03-16T09:58:57.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.975+0000 7ff02e200700 1 -- start start 2024-03-16T09:58:57.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.975+0000 7ff02e200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:57.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.976+0000 7ff02e200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810c320 con 0x7ff028102e30 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.976+0000 7ff027400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.976+0000 7ff027400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41922/0 (socket says 172.21.15.39:41922) 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.976+0000 7ff027400700 1 -- 172.21.15.39:0/3405301427 learned_addr learned my addr 172.21.15.39:0/3405301427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.976+0000 7ff027400700 1 -- 172.21.15.39:0/3405301427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028102980 con 0x7ff028102e30 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff027400700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff018000f80 tx=0x7ff01800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018020020 con 0x7ff028102e30 2024-03-16T09:58:57.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018018040 con 0x7ff028102e30 2024-03-16T09:58:57.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff028101030 con 0x7ff028102e30 2024-03-16T09:58:57.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff018017070 con 0x7ff028102e30 2024-03-16T09:58:57.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.977+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281014f0 con 0x7ff028102e30 2024-03-16T09:58:57.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.978+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7ff018023020 con 0x7ff028102e30 2024-03-16T09:58:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.978+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028069fa0 con 0x7ff028102e30 2024-03-16T09:58:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.979+0000 7ff024c00700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 0x7ff00c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:58:57.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.979+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff01800f050 con 0x7ff028102e30 2024-03-16T09:58:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.979+0000 7ff01ea00700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 0x7ff00c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:58:57.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.980+0000 7ff01ea00700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 0x7ff00c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff010006fd0 tx=0x7ff010006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:58:57.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:57.985+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff01802a020 con 0x7ff028102e30 2024-03-16T09:58:58.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.219+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff028101fc0 con 0x7ff028102e30 2024-03-16T09:58:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.297+0000 7ff024c00700 1 -- 172.21.15.39:0/3405301427 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff01802c540 con 0x7ff028102e30 2024-03-16T09:58:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 msgr2=0x7ff00c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 0x7ff00c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff010006fd0 tx=0x7ff010006e40 comp rx=0 tx=0).stop 2024-03-16T09:58:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 msgr2=0x7ff028100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:58:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff018000f80 tx=0x7ff01800bd20 comp rx=0 tx=0).stop 2024-03-16T09:58:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 shutdown_connections 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff00c039060 0x7ff00c03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 --2- 172.21.15.39:0/3405301427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff028102e30 0x7ff028100a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.301+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 >> 172.21.15.39:0/3405301427 conn(0x7ff0280777d0 msgr2=0x7ff028103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.302+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 shutdown_connections 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:58:58.302+0000 7ff02e200700 1 -- 172.21.15.39:0/3405301427 wait complete. 2024-03-16T09:58:58.304 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:58:58.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:58:58.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:58:58.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3405301427' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T09:58:59.091 INFO:tasks.cephadm:Adding mon.a on smithi039 2024-03-16T09:58:59.091 INFO:tasks.cephadm:Adding mon.b on smithi105 2024-03-16T09:58:59.091 INFO:tasks.cephadm:Adding mon.c on smithi194 2024-03-16T09:58:59.091 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c' 2024-03-16T09:58:59.317 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:58:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:58:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:58:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3405301427' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T09:58:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:58:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:58:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:58:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:00.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 -- 172.21.15.194:0/2991601817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 msgr2=0x7fca501011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:00.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 --2- 172.21.15.194:0/2991601817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501011f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca44009910 tx=0x7fca44009c20 comp rx=0 tx=0).stop 2024-03-16T09:59:00.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 -- 172.21.15.194:0/2991601817 shutdown_connections 2024-03-16T09:59:00.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 --2- 172.21.15.194:0/2991601817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501011f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 -- 172.21.15.194:0/2991601817 >> 172.21.15.194:0/2991601817 conn(0x7fca500fc6c0 msgr2=0x7fca500feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 -- 172.21.15.194:0/2991601817 shutdown_connections 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.618+0000 7fca56be8700 1 -- 172.21.15.194:0/2991601817 wait complete. 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca56be8700 1 Processor -- start 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca56be8700 1 -- start start 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca56be8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:00.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca56be8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca5010b1e0 con 0x7fca50100e10 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca54984700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca54984700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.194:45340/0 (socket says 172.21.15.194:45340) 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.619+0000 7fca54984700 1 -- 172.21.15.194:0/2652265401 learned_addr learned my addr 172.21.15.194:0/2652265401 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.620+0000 7fca54984700 1 -- 172.21.15.194:0/2652265401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca50196c20 con 0x7fca50100e10 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.620+0000 7fca54984700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca44005850 tx=0x7fca44005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.621+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca44026070 con 0x7fca50100e10 2024-03-16T09:59:00.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.621+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca50196f10 con 0x7fca50100e10 2024-03-16T09:59:00.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.621+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5019afb0 con 0x7fca50100e10 2024-03-16T09:59:00.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.621+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca4400b6f0 con 0x7fca50100e10 2024-03-16T09:59:00.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.621+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca44030060 con 0x7fca50100e10 2024-03-16T09:59:00.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.622+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34005330 con 0x7fca50100e10 2024-03-16T09:59:00.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.622+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fca44029070 con 0x7fca50100e10 2024-03-16T09:59:00.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.622+0000 7fca4dffb700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 0x7fca4003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:00.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.623+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca4401e070 con 0x7fca50100e10 2024-03-16T09:59:00.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.623+0000 7fca4ffff700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 0x7fca4003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:00.625 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.624+0000 7fca4ffff700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 0x7fca4003b430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006fd0 tx=0x7fca3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:00.638 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.637+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca44013070 con 0x7fca50100e10 2024-03-16T09:59:00.842 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.841+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}) -- 0x7fca34000c80 con 0x7fca40038f70 2024-03-16T09:59:00.859 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.858+0000 7fca4dffb700 1 -- 172.21.15.194:0/2652265401 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fca34000c80 con 0x7fca40038f70 2024-03-16T09:59:00.860 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 msgr2=0x7fca4003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 0x7fca4003b430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006fd0 tx=0x7fca3c006e40 comp rx=0 tx=0).stop 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 msgr2=0x7fca501966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca44005850 tx=0x7fca44005030 comp rx=0 tx=0).stop 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 shutdown_connections 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fca40038f70 0x7fca4003b430 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 --2- 172.21.15.194:0/2652265401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca50100e10 0x7fca501966e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:00.862 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 >> 172.21.15.194:0/2652265401 conn(0x7fca500fc6c0 msgr2=0x7fca50103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:00.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 shutdown_connections 2024-03-16T09:59:00.863 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:00.860+0000 7fca56be8700 1 -- 172.21.15.194:0/2652265401 wait complete. 2024-03-16T09:59:01.430 DEBUG:teuthology.orchestra.run.smithi105:mon.b> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.b.service 2024-03-16T09:59:01.433 DEBUG:teuthology.orchestra.run.smithi194:mon.c> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.c.service 2024-03-16T09:59:01.435 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:59:01.436 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c;count:3 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:02.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:01 smithi039 ceph-mon[31510]: Deploying daemon mon.c on smithi194 2024-03-16T09:59:03.248 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:04.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:03 smithi039 ceph-mon[31510]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 -- 172.21.15.194:0/1483372527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 msgr2=0x7f7d68103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 --2- 172.21.15.194:0/1483372527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d68103210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009910 tx=0x7f7d64009c20 comp rx=0 tx=0).stop 2024-03-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 -- 172.21.15.194:0/1483372527 shutdown_connections 2024-03-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 --2- 172.21.15.194:0/1483372527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d68103210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 -- 172.21.15.194:0/1483372527 >> 172.21.15.194:0/1483372527 conn(0x7f7d680777d0 msgr2=0x7f7d680feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 -- 172.21.15.194:0/1483372527 shutdown_connections 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.151+0000 7f7d6f516700 1 -- 172.21.15.194:0/1483372527 wait complete. 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.152+0000 7f7d6f516700 1 Processor -- start 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.152+0000 7f7d6f516700 1 -- start start 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.152+0000 7f7d6f516700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.152+0000 7f7d6f516700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6810c320 con 0x7f7d68102e30 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.152+0000 7f7d6d2b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.153+0000 7f7d6d2b2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.194:44100/0 (socket says 172.21.15.194:44100) 2024-03-16T09:59:05.155 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.153+0000 7f7d6d2b2700 1 -- 172.21.15.194:0/3480257349 learned_addr learned my addr 172.21.15.194:0/3480257349 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.153+0000 7f7d6d2b2700 1 -- 172.21.15.194:0/3480257349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6819daf0 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.153+0000 7f7d6d2b2700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d64006010 tx=0x7f7d64005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.154+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d64026070 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.154+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6819dd80 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.154+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6400bc30 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.154+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d640039a0 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.154+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d68197680 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.155+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d68069fa0 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.155+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7d64029070 con 0x7f7d68102e30 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.155+0000 7f7d5e7fc700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 0x7f7d5403f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.156+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d6401e070 con 0x7f7d68102e30 2024-03-16T09:59:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.156+0000 7f7d6cab1700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 0x7f7d5403f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.157+0000 7f7d6cab1700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 0x7f7d5403f890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:05.174 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.172+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d64013070 con 0x7f7d68102e30 2024-03-16T09:59:05.469 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.467+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7d68063930 con 0x7f7d68102e30 2024-03-16T09:59:05.474 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.471+0000 7f7d5e7fc700 1 -- 172.21.15.194:0/3480257349 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f7d68063930 con 0x7f7d68102e30 2024-03-16T09:59:05.475 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:59:05.475 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","modified":"2024-03-16T09:56:58.735570Z","created":"2024-03-16T09:56:58.735570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T09:59:05.478 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 msgr2=0x7f7d5403f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:05.478 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 0x7f7d5403f890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d58006fd0 tx=0x7f7d58006e40 comp rx=0 tx=0).stop 2024-03-16T09:59:05.478 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 msgr2=0x7f7d6819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d64006010 tx=0x7f7d64005030 comp rx=0 tx=0).stop 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 shutdown_connections 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7d5403d3d0 0x7f7d5403f890 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 --2- 172.21.15.194:0/3480257349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d68102e30 0x7f7d6819d5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 >> 172.21.15.194:0/3480257349 conn(0x7f7d680777d0 msgr2=0x7f7d68103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 shutdown_connections 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:05.477+0000 7f7d6f516700 1 -- 172.21.15.194:0/3480257349 wait complete. 2024-03-16T09:59:05.479 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-03-16T09:59:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:05 smithi194 systemd[1]: Starting Ceph mon.c for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T09:59:06.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:05 smithi039 ceph-mon[31510]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:05 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/3480257349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 podman[36425]: 2024-03-16 09:59:06.13982198 +0000 UTC m=+0.018212540 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 podman[36425]: 2024-03-16 09:59:06.239891767 +0000 UTC m=+0.118282316 container create 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 podman[36425]: 2024-03-16 09:59:06.438297224 +0000 UTC m=+0.316687804 container init 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 podman[36425]: 2024-03-16 09:59:06.441406108 +0000 UTC m=+0.319796657 container start 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T09:59:06.504 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pidfile_write: ignore empty --pid-file 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: load: jerasure load: lrc 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: RocksDB version: 7.9.2 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Git sha 0 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: DB SUMMARY 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: DB Session ID: CP9HNDTY4F130C7T3182 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: CURRENT file: CURRENT 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-16T09:59:06.505 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.error_if_exists: 0 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.create_if_missing: 0 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.paranoid_checks: 1 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.env: 0x55f4a4e4dc60 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.info_log: 0x55f4a58d9ac0 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.statistics: (nil) 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.use_fsync: 0 2024-03-16T09:59:06.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_log_file_size: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_fallocate: 1 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.use_direct_reads: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.db_log_dir: 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.wal_dir: 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T09:59:06.507 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.write_buffer_manager: 0x55f4a58ac1e0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.unordered_write: 0 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T09:59:06.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.row_cache: None 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.wal_filter: None 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.two_write_queues: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.wal_compression: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.atomic_flush: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.log_readahead_size: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T09:59:06.509 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_background_jobs: 2 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_background_compactions: -1 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_subcompactions: 1 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T09:59:06.510 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T09:59:06.511 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_open_files: -1 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_background_flushes: -1 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Compression algorithms supported: 2024-03-16T09:59:06.512 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kZSTD supported: 0 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kXpressCompression supported: 0 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kZlibCompression supported: 1 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kSnappyCompression supported: 1 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kLZ4Compression supported: 1 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: kBZip2Compression supported: 0 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.merge_operator: 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_filter: None 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T09:59:06.513 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f4a58d9bc0) 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: cache_index_and_filter_blocks: 1 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: pin_top_level_index_and_filter: 1 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_type: 0 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: data_block_index_type: 0 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_shortening: 1 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: checksum: 4 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: no_block_cache: 0 2024-03-16T09:59:06.514 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache: 0x55f4a58df090 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_name: BinnedLRUCache 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_options: 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: capacity : 536870912 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: num_shard_bits : 4 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: strict_capacity_limit : 0 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: high_pri_pool_ratio: 0.000 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_cache_compressed: (nil) 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: persistent_cache: (nil) 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_size: 4096 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_size_deviation: 10 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_restart_interval: 16 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: index_block_restart_interval: 1 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: metadata_block_size: 4096 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: partition_filters: 0 2024-03-16T09:59:06.515 INFO:journalctl@ceph.mon.c.smithi194.stdout: use_delta_encoding: 1 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: filter_policy: bloomfilter 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: whole_key_filtering: 1 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: verify_compression: 0 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: read_amp_bytes_per_bit: 0 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: format_version: 5 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: enable_index_compression: 1 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: block_align: 0 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: max_auto_readahead_size: 262144 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: prepopulate_block_cache: 0 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: initial_auto_readahead_size: 8192 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T09:59:06.516 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression: NoCompression 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.num_levels: 7 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T09:59:06.517 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T09:59:06.518 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T09:59:06.519 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T09:59:06.520 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.table_properties_collectors: 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.inplace_update_support: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.bloom_locality: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.max_successive_merges: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.ttl: 2592000 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T09:59:06.521 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enable_blob_files: false 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.min_blob_size: 0 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 01beb59e-1ca7-4cb5-8ade-b118dca76605 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583146471713, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-16T09:59:06.522 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583146472302, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710583146, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "01beb59e-1ca7-4cb5-8ade-b118dca76605", "db_session_id": "CP9HNDTY4F130C7T3182", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710583146472407, "job": 1, "event": "recovery_finished"} 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f4a59d2000 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: DB pointer 0x55f4a59bc000 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** DB Stats ** 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:59:06.523 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** Compaction Stats [default] ** 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** Compaction Stats [default] ** 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T09:59:06.524 INFO:journalctl@ceph.mon.c.smithi194.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Block cache BinnedLRUCache@0x55f4a58df090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-16T09:59:06.525 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: using public_addr v2:172.21.15.194:0/0 -> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: starting mon.c rank -1 at public addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] at bind addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(???) e0 preinit fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).mds e1 new map 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: e1 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: legacy client fscid: -1 2024-03-16T09:59:06.526 INFO:journalctl@ceph.mon.c.smithi194.stdout: 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout: No filesystems configured 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Deploying cephadm binary to smithi105 2024-03-16T09:59:06.527 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Added host smithi105 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.528 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.529 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Deploying cephadm binary to smithi194 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Added host smithi194 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.530 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3405301427' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T09:59:06.531 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3405301427' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c;count:3 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:06.532 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: Deploying daemon mon.c on smithi194 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/3480257349' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:06.533 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 ceph-mon[36445]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-16T09:59:06.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 bash[36425]: 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a 2024-03-16T09:59:06.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:06 smithi194 systemd[1]: Started Ceph mon.c for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:59:07.029 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:59:07.029 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:59:07.263 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:59:11.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 -- 172.21.15.194:0/1432965238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 msgr2=0x7fbb30005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:11.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 --2- 172.21.15.194:0/1432965238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb30005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005fa0 tx=0x7fbb3c00ffc0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 -- 172.21.15.194:0/1432965238 shutdown_connections 2024-03-16T09:59:11.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 --2- 172.21.15.194:0/1432965238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb30005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 -- 172.21.15.194:0/1432965238 >> 172.21.15.194:0/1432965238 conn(0x7fbb440fc770 msgr2=0x7fbb440feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:11.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 -- 172.21.15.194:0/1432965238 shutdown_connections 2024-03-16T09:59:11.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.617+0000 7fbb4b403700 1 -- 172.21.15.194:0/1432965238 wait complete. 2024-03-16T09:59:11.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 Processor -- start 2024-03-16T09:59:11.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 -- start start 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb4419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44108d10 con 0x7fbb44100ee0 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4b403700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44108b90 con 0x7fbb4419cac0 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4899e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.618+0000 7fbb4899e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:45172/0 (socket says 172.21.15.194:45172) 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4899e700 1 -- 172.21.15.194:0/2784195290 learned_addr learned my addr 172.21.15.194:0/2784195290 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:11.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4899e700 1 -- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 msgr2=0x7fbb44196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4899e700 1 -- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 msgr2=0x7fbb44196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4899e700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4899e700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4919f700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb4419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4919f700 1 -- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 msgr2=0x7fbb44196650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.619+0000 7fbb4919f700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb4919f700 1 -- 172.21.15.194:0/2784195290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4419d040 con 0x7fbb44100ee0 2024-03-16T09:59:11.622 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb4919f700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb4419c580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:11.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbb3c0053b0 con 0x7fbb44100ee0 2024-03-16T09:59:11.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb44196bf0 con 0x7fbb44100ee0 2024-03-16T09:59:11.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb44197150 con 0x7fbb44100ee0 2024-03-16T09:59:11.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb3c005550 con 0x7fbb44100ee0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.620+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbb3c011630 con 0x7fbb44100ee0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.622+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb441023c0 con 0x7fbb44100ee0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.622+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fbb3c01b430 con 0x7fbb44100ee0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.622+0000 7fbb3a7fc700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 0x7fbb2803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.622+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbb3c04e790 con 0x7fbb44100ee0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.623+0000 7fbb4899e700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 0x7fbb2803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:11.624 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.623+0000 7fbb4899e700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 0x7fbb2803b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009180 tx=0x7fbb34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:11.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.638+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbb3c022070 con 0x7fbb44100ee0 2024-03-16T09:59:11.914 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.913+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbb44197430 con 0x7fbb44100ee0 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: mon.a calling monitor election 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.915 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: mon.c calling monitor election 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: monmap epoch 2 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: last_changed 2024-03-16T09:59:06.509569+0000 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:11.916 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: min_mon_release 19 (squid) 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: election_strategy: 1 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: fsmap 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: mgrmap e13: x(active, since 59s) 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: overall HEALTH_OK 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:11.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:11.918 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.914+0000 7fbb3a7fc700 1 -- 172.21.15.194:0/2784195290 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fbb3c011880 con 0x7fbb44100ee0 2024-03-16T09:59:11.918 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:59:11.918 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","modified":"2024-03-16T09:59:06.509569Z","created":"2024-03-16T09:56:58.735570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 msgr2=0x7fbb2803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 0x7fbb2803b220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009180 tx=0x7fbb34008040 comp rx=0 tx=0).stop 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 msgr2=0x7fbb4419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb4419c580 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c004d60 comp rx=0 tx=0).stop 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 shutdown_connections 2024-03-16T09:59:11.919 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbb28038d60 0x7fbb2803b220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb44100ee0 0x7fbb4419c580 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 --2- 172.21.15.194:0/2784195290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb4419cac0 0x7fbb44196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 >> 172.21.15.194:0/2784195290 conn(0x7fbb440fc770 msgr2=0x7fbb440fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 shutdown_connections 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:11.917+0000 7fbb4b403700 1 -- 172.21.15.194:0/2784195290 wait complete. 2024-03-16T09:59:11.920 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-03-16T09:59:12.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:12.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:59:12.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: mon.a calling monitor election 2024-03-16T09:59:12.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: mon.c calling monitor election 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: monmap epoch 2 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: last_changed 2024-03-16T09:59:06.509569+0000 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:12.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: min_mon_release 19 (squid) 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: election_strategy: 1 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: fsmap 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: mgrmap e13: x(active, since 59s) 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: overall HEALTH_OK 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:12.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:12 smithi194 ceph-mon[36445]: Deploying daemon mon.b on smithi105 2024-03-16T09:59:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:12 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/2784195290' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:12 smithi039 ceph-mon[31510]: Deploying daemon mon.b on smithi105 2024-03-16T09:59:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:12 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/2784195290' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:13.518 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:59:13.519 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:59:13.628 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 09:59:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T09:59:13.510+0000 7ff97bc00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-16T09:59:13.761 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:59:13.883 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:13 smithi194 ceph-mon[36445]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:14.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:13 smithi039 ceph-mon[31510]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:15.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 -- 172.21.15.194:0/311152074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e4102010 msgr2=0x7f67e41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:15.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 --2- 172.21.15.194:0/311152074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e4102010 0x7f67e41023f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009910 tx=0x7f67d4009c20 comp rx=0 tx=0).stop 2024-03-16T09:59:15.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 -- 172.21.15.194:0/311152074 shutdown_connections 2024-03-16T09:59:15.209 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 --2- 172.21.15.194:0/311152074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e4102010 0x7f67e41023f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:15.210 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 -- 172.21.15.194:0/311152074 >> 172.21.15.194:0/311152074 conn(0x7f67e40fd8a0 msgr2=0x7f67e40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:15.210 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 -- 172.21.15.194:0/311152074 shutdown_connections 2024-03-16T09:59:15.210 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 -- 172.21.15.194:0/311152074 wait complete. 2024-03-16T09:59:15.214 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.207+0000 7f67ec253700 1 Processor -- start 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67ec253700 1 -- start start 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67ec253700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67ec253700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 0x7f67e41978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67ec253700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67e410c3e0 con 0x7f67e419dbf0 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67ec253700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67e410c260 con 0x7f67e4102010 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:15.215 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:56448/0 (socket says 172.21.15.194:56448) 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 -- 172.21.15.194:0/1843886963 learned_addr learned my addr 172.21.15.194:0/1843886963 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 -- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 msgr2=0x7f67e419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 -- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 msgr2=0x7f67e419d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e9fef700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:59:15.216 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.208+0000 7f67e97ee700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 0x7f67e41978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.209+0000 7f67e97ee700 1 -- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 msgr2=0x7f67e419d6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.209+0000 7f67e97ee700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.209+0000 7f67e97ee700 1 -- 172.21.15.194:0/1843886963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e419e230 con 0x7f67e419dbf0 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.209+0000 7f67e97ee700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 0x7f67e41978d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b4d0 tx=0x7f67e000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f67e0011630 con 0x7f67e419dbf0 2024-03-16T09:59:15.217 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e4197e70 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41983d0 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e0011cb0 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f67e00193b0 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.210+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8005330 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.211+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f67e0018020 con 0x7f67e419dbf0 2024-03-16T09:59:15.218 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.211+0000 7f67daffd700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 0x7f67d003b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:15.219 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.211+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f67e004bd20 con 0x7f67e419dbf0 2024-03-16T09:59:15.219 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.212+0000 7f67e9fef700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 0x7f67d003b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:15.219 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.213+0000 7f67e9fef700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 0x7f67d003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:15.229 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.228+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f67e001f020 con 0x7f67e419dbf0 2024-03-16T09:59:15.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.504+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f67c80058f0 con 0x7f67e419dbf0 2024-03-16T09:59:15.506 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.505+0000 7f67daffd700 1 -- 172.21.15.194:0/1843886963 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f67e0011a60 con 0x7f67e419dbf0 2024-03-16T09:59:15.506 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:59:15.506 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","modified":"2024-03-16T09:59:06.509569Z","created":"2024-03-16T09:56:58.735570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:59:15.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 msgr2=0x7f67d003b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:15.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 0x7f67d003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d400b580 comp rx=0 tx=0).stop 2024-03-16T09:59:15.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 msgr2=0x7f67e41978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 0x7f67e41978d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67e000b4d0 tx=0x7f67e000b890 comp rx=0 tx=0).stop 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 shutdown_connections 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f67d0038d60 0x7f67d003b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67e4102010 0x7f67e419d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 --2- 172.21.15.194:0/1843886963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67e419dbf0 0x7f67e41978d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 >> 172.21.15.194:0/1843886963 conn(0x7f67e40fd8a0 msgr2=0x7f67e41039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 shutdown_connections 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:15.507+0000 7f67ec253700 1 -- 172.21.15.194:0/1843886963 wait complete. 2024-03-16T09:59:15.509 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-03-16T09:59:15.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:15 smithi194 ceph-mon[36445]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:15.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:15 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/1843886963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:15 smithi039 ceph-mon[31510]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:15 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/1843886963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:59:16.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:15 smithi105 systemd[1]: Started Ceph mon.b for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:59:17.062 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:59:17.062 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:59:17.304 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: mon.c calling monitor election 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: mon.a calling monitor election 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: monmap epoch 3 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: min_mon_release 19 (squid) 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: election_strategy: 1 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: fsmap 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: mgrmap e13: x(active, since 68s) 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:59:21.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:59:21.209 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: mon.b (rank 2) addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] is down (out of quorum) 2024-03-16T09:59:21.209 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.209 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: mon.c calling monitor election 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: mon.a calling monitor election 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: monmap epoch 3 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: min_mon_release 19 (squid) 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: election_strategy: 1 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: fsmap 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:21.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: mgrmap e13: x(active, since 68s) 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: mon.b (rank 2) addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] is down (out of quorum) 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.613 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 -- 172.21.15.194:0/2973847315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 msgr2=0x7f89a0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 --2- 172.21.15.194:0/2973847315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89a0005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f89ac005fa0 tx=0x7f89ac00ffc0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 -- 172.21.15.194:0/2973847315 shutdown_connections 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 --2- 172.21.15.194:0/2973847315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89a0005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 -- 172.21.15.194:0/2973847315 >> 172.21.15.194:0/2973847315 conn(0x7f89b80fc7e0 msgr2=0x7f89b80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 -- 172.21.15.194:0/2973847315 shutdown_connections 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.611+0000 7f89bd16e700 1 -- 172.21.15.194:0/2973847315 wait complete. 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 Processor -- start 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 -- start start 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89b819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:21.614 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8108d10 con 0x7f89b8100f30 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8108b90 con 0x7f89b8196d30 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.612+0000 7f89bd16e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8108e90 con 0x7f89b819cb60 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:56462/0 (socket says 172.21.15.194:56462) 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 -- 172.21.15.194:0/639415247 learned_addr learned my addr 172.21.15.194:0/639415247 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b759e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b6d9d700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89b819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b759e700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 msgr2=0x7f89b81971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b759e700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 msgr2=0x7f89b81971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:59:21.617 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 msgr2=0x7f89b81971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b759e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b759e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 msgr2=0x7f89b819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89b819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 -- 172.21.15.194:0/639415247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b806a4b0 con 0x7f89b819cb60 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b6d9d700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89b819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.613+0000 7f89b659c700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b780 tx=0x7f89a800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8004230 con 0x7f89b819cb60 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b806a7a0 con 0x7f89b819cb60 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a80043d0 con 0x7f89b819cb60 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011c60 con 0x7f89b819cb60 2024-03-16T09:59:21.618 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b806ace0 con 0x7f89b819cb60 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.614+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f89a801a400 con 0x7f89b819cb60 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.615+0000 7f899ffff700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 0x7f89a403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.615+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f89a804cd30 con 0x7f89b819cb60 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.615+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b8102410 con 0x7f89b819cb60 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.615+0000 7f89b6d9d700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 0x7f89a403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:21.619 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.616+0000 7f89b6d9d700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 0x7f89a403b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00fcb0 tx=0x7f89ac005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:21.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.620+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f89a8020500 con 0x7f89b819cb60 2024-03-16T09:59:21.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.935+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f89b806afa0 con 0x7f89b819cb60 2024-03-16T09:59:21.936 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:21 smithi194 ceph-mon[36445]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:21.936 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:21.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:21.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:21.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.935+0000 7f899ffff700 1 -- 172.21.15.194:0/639415247 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f89a80202a0 con 0x7f89b819cb60 2024-03-16T09:59:21.938 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:59:21.938 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":3,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","modified":"2024-03-16T09:59:15.748663Z","created":"2024-03-16T09:56:58.735570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.937+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 msgr2=0x7f89a403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.937+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 0x7f89a403b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00fcb0 tx=0x7f89ac005920 comp rx=0 tx=0).stop 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.937+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 msgr2=0x7f89b81967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.937+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b780 tx=0x7f89a800bb40 comp rx=0 tx=0).stop 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 shutdown_connections 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89a4039450 0x7f89a403b910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.939 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b8196d30 0x7f89b81971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89b8100f30 0x7f89b819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 --2- 172.21.15.194:0/639415247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89b819cb60 0x7f89b81967f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 >> 172.21.15.194:0/639415247 conn(0x7f89b80fc7e0 msgr2=0x7f89b8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 shutdown_connections 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:21.938+0000 7f89bd16e700 1 -- 172.21.15.194:0/639415247 wait complete. 2024-03-16T09:59:21.940 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 3 2024-03-16T09:59:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:21 smithi039 ceph-mon[31510]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:22.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:22.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.069 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-16T09:59:23.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: mon.c calling monitor election 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: mon.a calling monitor election 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:59:23.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: monmap epoch 3 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:23.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: min_mon_release 19 (squid) 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: election_strategy: 1 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: fsmap 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: mgrmap e13: x(active, since 68s) 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: mon.b (rank 2) addr [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] is down (out of quorum) 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:23.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:59:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mon.b calling monitor election 2024-03-16T09:59:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T09:59:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T09:59:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mon.b calling monitor election 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mon.c calling monitor election 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mon.a calling monitor election 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: monmap epoch 3 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: min_mon_release 19 (squid) 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: election_strategy: 1 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: fsmap 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: mgrmap e13: x(active, since 70s) 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: overall HEALTH_OK 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mon.b calling monitor election 2024-03-16T09:59:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T09:59:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mon.b calling monitor election 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mon.c calling monitor election 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mon.a calling monitor election 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: monmap epoch 3 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: min_mon_release 19 (squid) 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: election_strategy: 1 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:23.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: fsmap 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: mgrmap e13: x(active, since 70s) 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: overall HEALTH_OK 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mon.b calling monitor election 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mon.b calling monitor election 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mon.c calling monitor election 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mon.a calling monitor election 2024-03-16T09:59:23.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: monmap epoch 3 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: last_changed 2024-03-16T09:59:15.748663+0000 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: created 2024-03-16T09:56:58.735570+0000 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: min_mon_release 19 (squid) 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: election_strategy: 1 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: 2: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: fsmap 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: mgrmap e13: x(active, since 70s) 2024-03-16T09:59:23.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: overall HEALTH_OK 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:23.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:24.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:24.860 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:24.860 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:24.860 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.860 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:24.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:25 smithi039 ceph-mon[31510]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:25 smithi039 ceph-mon[31510]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:59:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:25 smithi039 ceph-mon[31510]: Reconfiguring daemon mon.a on smithi039 2024-03-16T09:59:25.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 -- 172.21.15.39:0/170484970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc101f00 msgr2=0x7f7ecc1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/170484970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc101f00 0x7f7ecc1022e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00b0a0 tx=0x7f7ebc00b3b0 comp rx=0 tx=0).stop 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 -- 172.21.15.39:0/170484970 shutdown_connections 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/170484970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc101f00 0x7f7ecc1022e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 -- 172.21.15.39:0/170484970 >> 172.21.15.39:0/170484970 conn(0x7f7ecc0fd7b0 msgr2=0x7f7ecc0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 -- 172.21.15.39:0/170484970 shutdown_connections 2024-03-16T09:59:25.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.770+0000 7f7ed1a00700 1 -- 172.21.15.39:0/170484970 wait complete. 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.771+0000 7f7ed1a00700 1 Processor -- start 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 -- start start 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc19fe10 0x7f7ecc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ecc199fd0 0x7f7ecc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10c290 con 0x7f7ecc19fe10 2024-03-16T09:59:25.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10c110 con 0x7f7ecc199fd0 2024-03-16T09:59:25.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.772+0000 7f7ed1a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10c410 con 0x7f7ecc101f00 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ec2000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc19fe10 0x7f7ecc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:42766/0 (socket says 172.21.15.39:42766) 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 -- 172.21.15.39:0/2378952457 learned_addr learned my addr 172.21.15.39:0/2378952457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecb400700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ecc199fd0 0x7f7ecc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 -- 172.21.15.39:0/2378952457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ecc199fd0 msgr2=0x7f7ecc19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ecc199fd0 0x7f7ecc19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 -- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc19fe10 msgr2=0x7f7ecc199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc19fe10 0x7f7ecc199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 -- 172.21.15.39:0/2378952457 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1a67a0 con 0x7f7ecc101f00 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.773+0000 7f7ecaa00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0062a0 tx=0x7f7ebc003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.774+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc022070 con 0x7f7ecc101f00 2024-03-16T09:59:25.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.774+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc0055a0 con 0x7f7ecc101f00 2024-03-16T09:59:25.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.774+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc017070 con 0x7f7ecc101f00 2024-03-16T09:59:25.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.774+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1a6970 con 0x7f7ecc101f00 2024-03-16T09:59:25.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.774+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a6f00 con 0x7f7ecc101f00 2024-03-16T09:59:25.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.775+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7ebc01b070 con 0x7f7ecc101f00 2024-03-16T09:59:25.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.775+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc063900 con 0x7f7ecc101f00 2024-03-16T09:59:25.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.776+0000 7f7ec3e00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 0x7f7eb003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:25.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.776+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ebc00f020 con 0x7f7ecc101f00 2024-03-16T09:59:25.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.776+0000 7f7ec2000700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 0x7f7eb003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:25.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.777+0000 7f7ec2000700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 0x7f7eb003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009ad0 tx=0x7f7eb4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:25.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:25.781+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7ebc005750 con 0x7f7ecc101f00 2024-03-16T09:59:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:25 smithi105 ceph-mon[36559]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:25 smithi105 ceph-mon[36559]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:59:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:25 smithi105 ceph-mon[36559]: Reconfiguring daemon mon.a on smithi039 2024-03-16T09:59:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:25 smithi194 ceph-mon[36445]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:25 smithi194 ceph-mon[36445]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:59:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:25 smithi194 ceph-mon[36445]: Reconfiguring daemon mon.a on smithi039 2024-03-16T09:59:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.049+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7ecc1a7130 con 0x7f7ecc101f00 2024-03-16T09:59:26.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.050+0000 7f7ec3e00700 1 -- 172.21.15.39:0/2378952457 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f7ebc01ab40 con 0x7f7ecc101f00 2024-03-16T09:59:26.051 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:26.051 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-03-16T09:59:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T09:59:26.051 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 msgr2=0x7f7eb003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 0x7f7eb003b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009ad0 tx=0x7f7eb4009210 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 msgr2=0x7f7ecc19f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0062a0 tx=0x7f7ebc003f70 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 shutdown_connections 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7eb0039450 0x7f7eb003b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ecc199fd0 0x7f7ecc19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ecc101f00 0x7f7ecc19f8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 --2- 172.21.15.39:0/2378952457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ecc19fe10 0x7f7ecc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 >> 172.21.15.39:0/2378952457 conn(0x7f7ecc0fd7b0 msgr2=0x7f7ecc074500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 shutdown_connections 2024-03-16T09:59:26.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:26.053+0000 7f7ed1a00700 1 -- 172.21.15.39:0/2378952457 wait complete. 2024-03-16T09:59:26.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2378952457' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:26.785 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-16T09:59:26.785 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:59:26.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:59:26.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2378952457' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:26.872 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:59:26.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:59:26.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2378952457' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:26.983 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:59:26.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:59:27.018 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:59:27.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:59:27.089 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:59:27.089 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:59:27.124 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:59:27.125 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:59:27.194 INFO:tasks.cephadm:Adding mgr.x on smithi039 2024-03-16T09:59:27.194 INFO:tasks.cephadm:Adding mgr.y on smithi105 2024-03-16T09:59:27.195 INFO:tasks.cephadm:Adding mgr.z on smithi194 2024-03-16T09:59:27.195 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi039=x;smithi105=y;smithi194=z' 2024-03-16T09:59:27.468 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:27.870 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:28.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:28.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:28.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:28.879 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:28 smithi194 ceph-mon[36445]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:59:28.879 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:28 smithi194 ceph-mon[36445]: Reconfiguring daemon mon.b on smithi105 2024-03-16T09:59:28.986 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 -- 172.21.15.194:0/575564519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f581028e0 msgr2=0x7f3f58104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 --2- 172.21.15.194:0/575564519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f581028e0 0x7f3f58104cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009910 tx=0x7f3f4c009c20 comp rx=0 tx=0).stop 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 -- 172.21.15.194:0/575564519 shutdown_connections 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 --2- 172.21.15.194:0/575564519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f581028e0 0x7f3f58104cd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 -- 172.21.15.194:0/575564519 >> 172.21.15.194:0/575564519 conn(0x7f3f580fc2b0 msgr2=0x7f3f580fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 -- 172.21.15.194:0/575564519 shutdown_connections 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.983+0000 7f3f5f127700 1 -- 172.21.15.194:0/575564519 wait complete. 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 Processor -- start 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 -- start start 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f58197b40 0x7f3f5819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 0x7f3f5819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:28.987 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58107e40 con 0x7f3f58197b40 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58107cc0 con 0x7f3f58197fc0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5f127700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58107fc0 con 0x7f3f581028e0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:53788/0 (socket says 172.21.15.194:53788) 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 -- 172.21.15.194:0/2195966159 learned_addr learned my addr 172.21.15.194:0/2195966159 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5d6c4700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 0x7f3f5819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 -- 172.21.15.194:0/2195966159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 msgr2=0x7f3f5819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 0x7f3f5819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 -- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f58197b40 msgr2=0x7f3f5819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f58197b40 0x7f3f5819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5cec3700 1 -- 172.21.15.194:0/2195966159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5819fa10 con 0x7f3f581028e0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.984+0000 7f3f5d6c4700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 0x7f3f5819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f5cec3700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005000 tx=0x7f3f4c0050e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:28.988 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c027070 con 0x7f3f581028e0 2024-03-16T09:59:28.989 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5819fca0 con 0x7f3f581028e0 2024-03-16T09:59:28.989 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c022470 con 0x7f3f581028e0 2024-03-16T09:59:28.989 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00f6c0 con 0x7f3f581028e0 2024-03-16T09:59:28.989 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.985+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581a0200 con 0x7f3f581028e0 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.986+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f3f4c02a070 con 0x7f3f581028e0 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.986+0000 7f3f55ffb700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 0x7f3f4803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.986+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f58063900 con 0x7f3f581028e0 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.986+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4c04e2c0 con 0x7f3f581028e0 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.987+0000 7f3f57fff700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 0x7f3f4803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:28.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.988+0000 7f3f57fff700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 0x7f3f4803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098e0 tx=0x7f3f44006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:28.993 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:28.992+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f4c013070 con 0x7f3f581028e0 2024-03-16T09:59:29.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:28 smithi039 ceph-mon[31510]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:59:29.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:28 smithi039 ceph-mon[31510]: Reconfiguring daemon mon.b on smithi105 2024-03-16T09:59:29.018 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:28 smithi105 ceph-mon[36559]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:59:29.018 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:28 smithi105 ceph-mon[36559]: Reconfiguring daemon mon.b on smithi105 2024-03-16T09:59:29.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.206+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi105=y;smithi194=z", "target": ["mon-mgr", ""]}) -- 0x7f3f580627a0 con 0x7f3f48039450 2024-03-16T09:59:29.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.224+0000 7f3f55ffb700 1 -- 172.21.15.194:0/2195966159 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3f580627a0 con 0x7f3f48039450 2024-03-16T09:59:29.225 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mgr update... 2024-03-16T09:59:29.227 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 msgr2=0x7f3f4803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:29.227 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 0x7f3f4803b910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f440098e0 tx=0x7f3f44006dd0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.227 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 msgr2=0x7f3f58197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c005000 tx=0x7f3f4c0050e0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 shutdown_connections 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3f48039450 0x7f3f4803b910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f58197fc0 0x7f3f5819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f581028e0 0x7f3f58197600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 --2- 172.21.15.194:0/2195966159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f58197b40 0x7f3f5819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 >> 172.21.15.194:0/2195966159 conn(0x7f3f580fc2b0 msgr2=0x7f3f580fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 shutdown_connections 2024-03-16T09:59:29.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:59:29.226+0000 7f3f5f127700 1 -- 172.21.15.194:0/2195966159 wait complete. 2024-03-16T09:59:29.762 DEBUG:teuthology.orchestra.run.smithi105:mgr.y> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.y.service 2024-03-16T09:59:29.764 DEBUG:teuthology.orchestra.run.smithi194:mgr.z> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.z.service 2024-03-16T09:59:29.767 INFO:tasks.cephadm:Deploying OSDs... 2024-03-16T09:59:29.767 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T09:59:29.767 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:59:29.826 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:59:29.826 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-03-16 09:58:27.943234139 +0000 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-03-16 09:52:20.904956428 +0000 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-03-16 09:52:20.904956428 +0000 2024-03-16T09:59:29.879 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-03-16 09:52:20.904956428 +0000 2024-03-16T09:59:29.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:59:29.898 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:29.898 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:29.898 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:29.898 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:59:29.898 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:29.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:29.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:29.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:59:29.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:29.975 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-03-16T09:59:29.975 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-03-16T09:59:29.975 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000597152 s, 857 kB/s 2024-03-16T09:59:29.980 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:59:30.040 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-03-16 09:58:28.041232215 +0000 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-03-16 09:52:21.500944774 +0000 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-03-16 09:52:21.500944774 +0000 2024-03-16T09:59:30.094 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-03-16 09:52:21.500944774 +0000 2024-03-16T09:59:30.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:59:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:59:30.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:30.192 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-03-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-03-16T09:59:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000472194 s, 1.1 MB/s 2024-03-16T09:59:30.197 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:59:30.258 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-03-16 09:58:28.107230918 +0000 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-03-16 09:52:22.162931829 +0000 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-03-16 09:52:22.162931829 +0000 2024-03-16T09:59:30.312 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-03-16 09:52:22.162931829 +0000 2024-03-16T09:59:30.312 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:59:30.407 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-03-16T09:59:30.407 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-03-16T09:59:30.407 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000408931 s, 1.3 MB/s 2024-03-16T09:59:30.412 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:59:30.473 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-03-16 09:58:28.166229760 +0000 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-03-16 09:52:22.890917594 +0000 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-03-16 09:52:22.890917594 +0000 2024-03-16T09:59:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-03-16 09:52:22.890917594 +0000 2024-03-16T09:59:30.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:59:30.621 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-03-16T09:59:30.621 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-03-16T09:59:30.621 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000446884 s, 1.1 MB/s 2024-03-16T09:59:30.625 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:59:30.684 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-16T09:59:30.684 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:59:30.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:59:30.702 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-16 09:59:15.211926524 +0000 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-16 09:51:03.199691153 +0000 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-16 09:51:03.199691153 +0000 2024-03-16T09:59:30.760 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-16 09:51:03.199691153 +0000 2024-03-16T09:59:30.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:59:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-16T09:59:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-16T09:59:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000225949 s, 2.3 MB/s 2024-03-16T09:59:30.830 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:59:30.887 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-16T09:59:30.944 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-16 09:59:15.211926524 +0000 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-16 09:51:03.569683719 +0000 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-16 09:51:03.569683719 +0000 2024-03-16T09:59:30.945 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-16 09:51:03.569683719 +0000 2024-03-16T09:59:30.945 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:59:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:30 smithi194 ceph-mon[36445]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi105=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:30 smithi194 ceph-mon[36445]: Saving service mgr spec with placement smithi039=x;smithi105=y;smithi194=z;count:3 2024-03-16T09:59:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:30 smithi039 ceph-mon[31510]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi105=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:30 smithi039 ceph-mon[31510]: Saving service mgr spec with placement smithi039=x;smithi105=y;smithi194=z;count:3 2024-03-16T09:59:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-16T09:59:31.012 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-16T09:59:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000256221 s, 2.0 MB/s 2024-03-16T09:59:31.014 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:59:31.071 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-16T09:59:31.072 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:30 smithi105 ceph-mon[36559]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi105=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:31.072 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:30 smithi105 ceph-mon[36559]: Saving service mgr spec with placement smithi039=x;smithi105=y;smithi194=z;count:3 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-16 09:59:15.212926505 +0000 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-16 09:51:03.904676989 +0000 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-16 09:51:03.904676989 +0000 2024-03-16T09:59:31.087 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-16 09:51:03.904676989 +0000 2024-03-16T09:59:31.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:59:31.153 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-16T09:59:31.153 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-16T09:59:31.153 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00025178 s, 2.0 MB/s 2024-03-16T09:59:31.154 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:59:31.210 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-16T09:59:31.265 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:59:31.265 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.265 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-16T09:59:31.265 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.265 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.266 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-16 09:59:15.212926505 +0000 2024-03-16T09:59:31.266 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-16 09:51:04.234670359 +0000 2024-03-16T09:59:31.266 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-16 09:51:04.234670359 +0000 2024-03-16T09:59:31.266 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-16 09:51:04.234670359 +0000 2024-03-16T09:59:31.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:59:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-16T09:59:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-16T09:59:31.331 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000175871 s, 2.9 MB/s 2024-03-16T09:59:31.332 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:59:31.388 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:59:31.388 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:59:31.403 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:59:31.403 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-03-16T09:59:31.458 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:59:31.458 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:59:05.545720933 +0000 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:51:27.718295859 +0000 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:51:27.718295859 +0000 2024-03-16T09:59:31.459 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:51:27.718295859 +0000 2024-03-16T09:59:31.459 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:59:31.525 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:59:31.525 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:59:31.525 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000280073 s, 1.8 MB/s 2024-03-16T09:59:31.526 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:59:31.581 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:59:05.545720933 +0000 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:51:28.047288677 +0000 2024-03-16T09:59:31.635 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:51:28.047288677 +0000 2024-03-16T09:59:31.636 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:51:28.047288677 +0000 2024-03-16T09:59:31.636 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:59:31.701 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:59:31.701 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:59:31.701 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000177882 s, 2.9 MB/s 2024-03-16T09:59:31.702 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:59:31.757 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-03-16T09:59:31.813 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:59:05.545720933 +0000 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:51:28.390281190 +0000 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:51:28.390281190 +0000 2024-03-16T09:59:31.814 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:51:28.390281190 +0000 2024-03-16T09:59:31.814 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:59:31.879 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:59:31.880 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:59:31.880 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000184275 s, 2.8 MB/s 2024-03-16T09:59:31.881 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:59:31.936 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:59:05.546720912 +0000 2024-03-16T09:59:31.990 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:51:28.735273658 +0000 2024-03-16T09:59:31.991 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:51:28.735273658 +0000 2024-03-16T09:59:31.991 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:51:28.735273658 +0000 2024-03-16T09:59:31.991 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:32.054 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:32.054 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:32.054 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.054 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:31 smithi194 ceph-mon[36445]: Deploying daemon mgr.z on smithi194 2024-03-16T09:59:32.055 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:59:32.055 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:59:32.055 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000165758 s, 3.1 MB/s 2024-03-16T09:59:32.056 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:59:32.111 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2024-03-16T09:59:32.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:32.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:32.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:32.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:32.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:31 smithi105 ceph-mon[36559]: Deploying daemon mgr.z on smithi194 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:32.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:32.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:32.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:32.199 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:31 smithi039 ceph-mon[31510]: Deploying daemon mgr.z on smithi194 2024-03-16T09:59:32.560 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:59:33.127 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:32 smithi194 systemd[1]: Starting Ceph mgr.z for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T09:59:33.382 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 podman[38445]: 2024-03-16 09:59:33.126118862 +0000 UTC m=+0.099151481 container create 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T09:59:33.382 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 podman[38445]: 2024-03-16 09:59:33.044801489 +0000 UTC m=+0.017834134 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:59:33.383 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 podman[38445]: 2024-03-16 09:59:33.353463113 +0000 UTC m=+0.326495721 container init 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD) 2024-03-16T09:59:33.383 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 podman[38445]: 2024-03-16 09:59:33.356217704 +0000 UTC m=+0.329250323 container start 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T09:59:33.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mgr[38464]: -- 172.21.15.194:0/2554045138 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b019c25a0 con 0x558b01996c00 2024-03-16T09:59:33.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 bash[38445]: 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e 2024-03-16T09:59:33.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 systemd[1]: Started Ceph mgr.z for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:59:34.111 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:34.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:34.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:34.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:33.909+0000 7f9eef4e3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:59:34.207 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:34.001+0000 7f9eef4e3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:59:34.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-mon[36445]: Deploying daemon mgr.y on smithi105 2024-03-16T09:59:34.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:34.638+0000 7f9eef4e3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:59:34.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:34.732+0000 7f9eef4e3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:59:34.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:34.874+0000 7f9eef4e3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:59:35.141 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:34 smithi105 ceph-mon[36559]: Deploying daemon mgr.y on smithi105 2024-03-16T09:59:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:34 smithi039 ceph-mon[31510]: Deploying daemon mgr.y on smithi105 2024-03-16T09:59:35.314 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:34.968+0000 7f9eef4e3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:59:35.314 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.049+0000 7f9eef4e3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:59:35.314 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.312+0000 7f9eef4e3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:59:35.365 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T09:59:35.365 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T09:59:35.365 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-03-16T09:59:35.365 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-03-16T09:59:35.366 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0389841 s, 269 MB/s 2024-03-16T09:59:35.366 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2024-03-16T09:59:35.366 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-03-16T09:59:35.599 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.420+0000 7f9eef4e3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:59:35.599 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.503+0000 7f9eef4e3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:59:35.599 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.597+0000 7f9eef4e3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:59:35.874 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:35.774+0000 7f9eef4e3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:59:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:35 smithi039 ceph-mon[31510]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:36.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-03-16T09:59:36.106 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:35 smithi105 ceph-mon[36559]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:36.106 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:35 smithi105 ceph-mgr[37956]: -- 172.21.15.105:0/2736277318 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b65bf85a0 con 0x555b65bcc800 2024-03-16T09:59:36.106 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:36 smithi105 bash[37937]: 09abc74760e283f2c7865a54b7fb65dbaa51f940bbb088f6f0866c4c6147b0ec 2024-03-16T09:59:36.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:35 smithi194 ceph-mon[36445]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:36.365 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:36 smithi105 systemd[1]: Started Ceph mgr.y for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T09:59:36.564 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:59:36.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:36.514+0000 7fd7718bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:59:36.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:36.611+0000 7fd7718bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:59:37.174 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:36.758+0000 7f9eef4e3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:59:37.174 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:36.893+0000 7f9eef4e3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:59:37.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:37.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:37.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.457 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:37.256+0000 7f9eef4e3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:59:37.457 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:37.348+0000 7f9eef4e3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.292+0000 7fd7718bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:59:37.536 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.389+0000 7fd7718bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:59:37.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.534+0000 7fd7718bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:59:37.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.629+0000 7fd7718bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:59:37.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.717+0000 7fd7718bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:59:37.957 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:37.547+0000 7f9eef4e3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:59:37.957 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:37.742+0000 7f9eef4e3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:59:38.267 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:37.983+0000 7fd7718bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:59:38.267 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:38.090+0000 7fd7718bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:59:38.268 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:38.173+0000 7fd7718bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:59:38.268 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:38.266+0000 7fd7718bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:59:38.287 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:38.026+0000 7f9eef4e3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:59:38.287 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:38.286+0000 7f9eef4e3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:59:38.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 -- 172.21.15.39:0/3240189908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 msgr2=0x7f5b5c103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:38.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 --2- 172.21.15.39:0/3240189908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c103970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099c0 tx=0x7f5b48009cd0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 -- 172.21.15.39:0/3240189908 shutdown_connections 2024-03-16T09:59:38.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 --2- 172.21.15.39:0/3240189908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 0x7f5b5c10a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 --2- 172.21.15.39:0/3240189908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 0x7f5b5c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 --2- 172.21.15.39:0/3240189908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c103970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 -- 172.21.15.39:0/3240189908 >> 172.21.15.39:0/3240189908 conn(0x7f5b5c0fd870 msgr2=0x7f5b5c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:59:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 -- 172.21.15.39:0/3240189908 shutdown_connections 2024-03-16T09:59:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.460+0000 7f5b61a00700 1 -- 172.21.15.39:0/3240189908 wait complete. 2024-03-16T09:59:38.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.462+0000 7f5b61a00700 1 Processor -- start 2024-03-16T09:59:38.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.462+0000 7f5b61a00700 1 -- start start 2024-03-16T09:59:38.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 0x7f5b5c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:38.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:38.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 0x7f5b5c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:38.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c10df50 con 0x7f5b5c102310 2024-03-16T09:59:38.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c10ddd0 con 0x7f5b5c103510 2024-03-16T09:59:38.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b61a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c10e0d0 con 0x7f5b5c103eb0 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b5a000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.463+0000 7f5b5aa00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 0x7f5b5c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:35680/0 (socket says 172.21.15.39:35680) 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 -- 172.21.15.39:0/157529175 learned_addr learned my addr 172.21.15.39:0/157529175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5b400700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 0x7f5b5c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 -- 172.21.15.39:0/157529175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 msgr2=0x7f5b5c19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 0x7f5b5c19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 -- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 msgr2=0x7f5b5c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 0x7f5b5c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 -- 172.21.15.39:0/157529175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c1a01a0 con 0x7f5b5c103510 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.464+0000 7f5b5a000700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b48003790 tx=0x7f5b4800bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.465+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48009e60 con 0x7f5b5c103510 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.465+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4800f460 con 0x7f5b5c103510 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.465+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c1a0430 con 0x7f5b5c103510 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.465+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48017650 con 0x7f5b5c103510 2024-03-16T09:59:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.465+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c1a08e0 con 0x7f5b5c103510 2024-03-16T09:59:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.466+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5c068f90 con 0x7f5b5c103510 2024-03-16T09:59:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.466+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f5b480177f0 con 0x7f5b5c103510 2024-03-16T09:59:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.467+0000 7f5b53400700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 0x7f5b4003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:59:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.467+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b4804d090 con 0x7f5b5c103510 2024-03-16T09:59:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.467+0000 7f5b5aa00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 0x7f5b4003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:59:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.468+0000 7f5b5aa00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 0x7f5b4003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c103370 tx=0x7f5b4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:59:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.472+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b48013070 con 0x7f5b5c103510 2024-03-16T09:59:38.615 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:38.439+0000 7fd7718bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:59:38.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:38.518+0000 7f9eef4e3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:59:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:38.715+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5b5c062710 con 0x7f5b40039200 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:38 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:39.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:38 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:59:39.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:38 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:39.312+0000 7f9eef4e3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:59:39.718 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:39.396+0000 7fd7718bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:59:39.718 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:39.524+0000 7fd7718bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: Reconfiguring daemon mgr.x on smithi039 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:59:39.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.979 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: Reconfiguring daemon mgr.x on smithi039 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:39.882+0000 7f9eef4e3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:59:39.980 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:39.979+0000 7f9eef4e3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: Reconfiguring daemon mgr.x on smithi039 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:39.852+0000 7fd7718bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:59:40.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:39.939+0000 7fd7718bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:59:40.456 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:59:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T09:59:40.193+0000 7f9eef4e3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:59:40.593 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:40.131+0000 7fd7718bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:59:40.593 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:40.313+0000 7fd7718bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:59:40.849 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:40.591+0000 7fd7718bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:59:40.849 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:40.847+0000 7fd7718bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:59:41.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:41.079+0000 7fd7718bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:59:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:41.161+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7f5b4802b7c0 con 0x7f5b5c103510 2024-03-16T09:59:41.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:41.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: Standby manager daemon z started 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:59:41.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:41 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: Standby manager daemon z started 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:59:41.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:41 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:59:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: Standby manager daemon z started 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:59:41.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/151548192' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:42.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:41.867+0000 7fd7718bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:59:42.433 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:42 smithi105 ceph-mon[36559]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-16T09:59:42.434 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:59:42.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:42 smithi194 ceph-mon[36445]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-16T09:59:42.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:59:42.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:42 smithi039 ceph-mon[31510]: mgrmap e14: x(active, since 89s), standbys: z 2024-03-16T09:59:42.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:59:42.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:42.432+0000 7fd7718bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:59:42.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:42.527+0000 7fd7718bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:59:42.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 09:59:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T09:59:42.738+0000 7fd7718bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:59:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T09:59:43.191+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mon.2 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f5b480163a0 con 0x7f5b5c103510 2024-03-16T09:59:43.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:43.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: Standby manager daemon y started 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3826187709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]': finished 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:59:43.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: Standby manager daemon y started 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3826187709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.479 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.480 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]': finished 2024-03-16T09:59:43.480 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:59:43.480 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: Standby manager daemon y started 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:59:43.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/318410762' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:59:43.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3826187709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]: dispatch 2024-03-16T09:59:43.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5"}]': finished 2024-03-16T09:59:43.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:59:43.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:59:44.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:44 smithi039 ceph-mon[31510]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-16T09:59:44.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:44 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:59:44.455 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:44 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2762199423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:59:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:44 smithi194 ceph-mon[36445]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-16T09:59:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:44 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:59:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:44 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2762199423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:59:44.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:44 smithi105 ceph-mon[36559]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-03-16T09:59:44.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:44 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:59:44.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:44 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2762199423' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:59:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:45 smithi194 ceph-mon[36445]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:45 smithi039 ceph-mon[31510]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:45 smithi105 ceph-mon[36559]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:47 smithi194 ceph-mon[36445]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:47.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:47 smithi039 ceph-mon[31510]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:47.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:47 smithi105 ceph-mon[36559]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:49 smithi039 ceph-mon[31510]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:49 smithi105 ceph-mon[36559]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:49 smithi194 ceph-mon[36445]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:51.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:51 smithi039 ceph-mon[31510]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:51.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:51 smithi105 ceph-mon[36559]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:51 smithi194 ceph-mon[36445]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:53 smithi039 ceph-mon[31510]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:59:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:53 smithi039 ceph-mon[31510]: Deploying daemon osd.0 on smithi039 2024-03-16T09:59:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:53 smithi105 ceph-mon[36559]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:59:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:53 smithi105 ceph-mon[36559]: Deploying daemon osd.0 on smithi039 2024-03-16T09:59:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:53 smithi194 ceph-mon[36445]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:59:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:59:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:53 smithi194 ceph-mon[36445]: Deploying daemon osd.0 on smithi039 2024-03-16T09:59:55.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:55 smithi039 ceph-mon[31510]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:55.508 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:55 smithi194 ceph-mon[36445]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:55.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:55 smithi105 ceph-mon[36559]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:57.432 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:57 smithi039 ceph-mon[31510]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:57.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:57 smithi105 ceph-mon[36559]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:57 smithi194 ceph-mon[36445]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 09:59:59 smithi039 ceph-mon[31510]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:59.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 09:59:59 smithi105 ceph-mon[36559]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:59:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:59:59 smithi194 ceph-mon[36445]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:00.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:00 smithi105 ceph-mon[36559]: overall HEALTH_OK 2024-03-16T10:00:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:00 smithi194 ceph-mon[36445]: overall HEALTH_OK 2024-03-16T10:00:00.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:00 smithi039 ceph-mon[31510]: overall HEALTH_OK 2024-03-16T10:00:01.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:01 smithi039 ceph-mon[31510]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:01.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:01.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:01.702 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:01.777 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:01 smithi105 ceph-mon[36559]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:01.778 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:01.778 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:01.778 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:01 smithi194 ceph-mon[36445]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:03.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:03 smithi194 ceph-mon[36445]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:03 smithi039 ceph-mon[31510]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:03 smithi105 ceph-mon[36559]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.917+0000 7f5b53400700 1 -- 172.21.15.39:0/157529175 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b5c062710 con 0x7f5b40039200 2024-03-16T10:00:04.919 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2024-03-16T10:00:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 msgr2=0x7f5b4003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 0x7f5b4003b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c103370 tx=0x7f5b4c009260 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 msgr2=0x7f5b5c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b48003790 tx=0x7f5b4800bee0 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 shutdown_connections 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b40039200 0x7f5b4003b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b5c103eb0 0x7f5b5c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b5c102310 0x7f5b5c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 --2- 172.21.15.39:0/157529175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b5c103510 0x7f5b5c198a80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.921+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 >> 172.21.15.39:0/157529175 conn(0x7f5b5c0fd870 msgr2=0x7f5b5c106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.922+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 shutdown_connections 2024-03-16T10:00:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:04.922+0000 7f5b61a00700 1 -- 172.21.15.39:0/157529175 wait complete. 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:04 smithi105 ceph-mon[36559]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:00:05.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:05.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:05.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:04 smithi194 ceph-mon[36445]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:04 smithi039 ceph-mon[31510]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:00:05.697 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.0.service 2024-03-16T10:00:05.701 INFO:tasks.cephadm:Deploying osd.1 on smithi105 with /dev/vg_nvme/lv_4... 2024-03-16T10:00:05.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T10:00:05.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:05.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:06.069 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:06.506 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0[46558]: 2024-03-16T10:00:06.384+0000 7fce5ac00700 -1 osd.0 0 waiting for initial osdmap 2024-03-16T10:00:06.507 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0[46558]: 2024-03-16T10:00:06.402+0000 7fce50a00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:00:07.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-mon[31510]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:00:07.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-mon[31510]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T10:00:07.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:06 smithi039 ceph-mon[31510]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' 2024-03-16T10:00:07.111 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:06 smithi105 ceph-mon[36559]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:00:07.111 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:06 smithi105 ceph-mon[36559]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T10:00:07.111 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.111 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.111 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:06 smithi105 ceph-mon[36559]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' 2024-03-16T10:00:07.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:06 smithi194 ceph-mon[36445]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:00:07.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:06 smithi194 ceph-mon[36445]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T10:00:07.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:06 smithi194 ceph-mon[36445]: from='osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956]' entity='osd.0' 2024-03-16T10:00:07.667 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T10:00:07.667 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T10:00:07.667 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-16T10:00:07.667 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-16T10:00:07.667 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113673 s, 922 MB/s 2024-03-16T10:00:07.668 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-16T10:00:07.976 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] boot 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T10:00:07.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] boot 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T10:00:08.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:00:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:00:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:00:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:08.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: osd.0 [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] boot 2024-03-16T10:00:08.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T10:00:08.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:00:08.277 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-03-16T10:00:08.510 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:00:09.675 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:09 smithi105 ceph-mon[36559]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:09.675 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:09 smithi105 ceph-mon[36559]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T10:00:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:09 smithi194 ceph-mon[36445]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:09 smithi194 ceph-mon[36445]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T10:00:09.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:09 smithi039 ceph-mon[31510]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:09.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:09 smithi039 ceph-mon[31510]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.976+0000 7fb692fe3700 1 -- 172.21.15.105:0/3705111562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 msgr2=0x7fb68c107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.976+0000 7fb692fe3700 1 --2- 172.21.15.105:0/3705111562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c107b50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb678009990 tx=0x7fb678009ca0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 -- 172.21.15.105:0/3705111562 shutdown_connections 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 --2- 172.21.15.105:0/3705111562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68c108090 0x7fb68c10a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 --2- 172.21.15.105:0/3705111562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 0x7fb68c105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 --2- 172.21.15.105:0/3705111562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c107b50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 -- 172.21.15.105:0/3705111562 >> 172.21.15.105:0/3705111562 conn(0x7fb68c0fc7c0 msgr2=0x7fb68c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 -- 172.21.15.105:0/3705111562 shutdown_connections 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 -- 172.21.15.105:0/3705111562 wait complete. 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 Processor -- start 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 -- start start 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.977+0000 7fb692fe3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 0x7fb68c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68c108090 0x7fb68c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10dfd0 con 0x7fb68c108090 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10de50 con 0x7fb68c105760 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10e150 con 0x7fb68c102e30 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37404/0 (socket says 172.21.15.105:37404) 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 -- 172.21.15.105:0/1039097927 learned_addr learned my addr 172.21.15.105:0/1039097927 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb690d7f700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 0x7fb68c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 -- 172.21.15.105:0/1039097927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 msgr2=0x7fb68c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 0x7fb68c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 -- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68c108090 msgr2=0x7fb68c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68c108090 0x7fb68c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 -- 172.21.15.105:0/1039097927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c1a02d0 con 0x7fb68c105760 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb68bfff700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb678005910 tx=0x7fb6780047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678027070 con 0x7fb68c105760 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678022470 con 0x7fb68c105760 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67800f6a0 con 0x7fb68c105760 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.978+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c1a0560 con 0x7fb68c105760 2024-03-16T10:00:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.979+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c1a0a90 con 0x7fb68c105760 2024-03-16T10:00:09.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.980+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68c068f90 con 0x7fb68c105760 2024-03-16T10:00:09.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.981+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7fb67802a070 con 0x7fb68c105760 2024-03-16T10:00:09.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.981+0000 7fb689ffb700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 0x7fb67c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:09.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.981+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb6780ce0c0 con 0x7fb68c105760 2024-03-16T10:00:09.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.982+0000 7fb690d7f700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 0x7fb67c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:09.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.983+0000 7fb690d7f700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 0x7fb67c0a4d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb680006fd0 tx=0x7fb680008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:09.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:09.985+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb678013070 con 0x7fb68c105760 2024-03-16T10:00:10.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:10.189+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb68c062710 con 0x7fb67c0a2880 2024-03-16T10:00:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:10 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:10.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:10 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:10.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:10 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:10 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:10 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:10.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:10 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:10.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:10 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:10.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:10 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:10.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:10 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi039 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 17091M 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:11.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi039 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 17091M 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:11.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:11.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.839 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi039 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 17091M 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:11.840 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/3242735797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]': finished 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:13.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:13 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/2798791568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/3242735797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]': finished 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:13.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:13 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/2798791568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/3242735797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]: dispatch 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "097d0219-dbb6-485b-a385-33b389bf02c7"}]': finished 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:13.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:13 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/2798791568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:15 smithi039 ceph-mon[31510]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:15 smithi105 ceph-mon[36559]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:15.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:15 smithi194 ceph-mon[36445]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:17 smithi039 ceph-mon[31510]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:17.777 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:17 smithi105 ceph-mon[36559]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:17.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:17 smithi194 ceph-mon[36445]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:19 smithi039 ceph-mon[31510]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:19 smithi105 ceph-mon[36559]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:19.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:19 smithi194 ceph-mon[36445]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:00:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:20 smithi039 ceph-mon[31510]: Deploying daemon osd.1 on smithi105 2024-03-16T10:00:20.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:00:20.817 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:20.818 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:20 smithi105 ceph-mon[36559]: Deploying daemon osd.1 on smithi105 2024-03-16T10:00:20.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:00:20.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:20.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:20 smithi194 ceph-mon[36445]: Deploying daemon osd.1 on smithi105 2024-03-16T10:00:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:21 smithi105 ceph-mon[36559]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:21 smithi039 ceph-mon[31510]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:21 smithi194 ceph-mon[36445]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:23.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:23 smithi105 ceph-mon[36559]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:23.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:23 smithi039 ceph-mon[31510]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:23 smithi194 ceph-mon[36445]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:26.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:25 smithi105 ceph-mon[36559]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:26.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:25 smithi194 ceph-mon[36445]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:26.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:25 smithi039 ceph-mon[31510]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:26.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:26.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:28.050 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:28.050 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.050 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.050 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:28.050 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:28.051 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:27 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:28.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:28.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:27 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:27 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:28.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.425+0000 7fb689ffb700 1 -- 172.21.15.105:0/1039097927 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb68c062710 con 0x7fb67c0a2880 2024-03-16T10:00:28.427 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 1 on host 'smithi105' 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 msgr2=0x7fb67c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 0x7fb67c0a4d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb680006fd0 tx=0x7fb680008040 comp rx=0 tx=0).stop 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 msgr2=0x7fb68c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb678005910 tx=0x7fb6780047b0 comp rx=0 tx=0).stop 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 shutdown_connections 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb67c0a2880 0x7fb67c0a4d40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:28.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68c108090 0x7fb68c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:28.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb68c102e30 0x7fb68c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:28.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 --2- 172.21.15.105:0/1039097927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb68c105760 0x7fb68c198bb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:28.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 >> 172.21.15.105:0/1039097927 conn(0x7fb68c0fc7c0 msgr2=0x7fb68c1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:28.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 shutdown_connections 2024-03-16T10:00:28.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:00:28.428+0000 7fb692fe3700 1 -- 172.21.15.105:0/1039097927 wait complete. 2024-03-16T10:00:29.029 DEBUG:teuthology.orchestra.run.smithi105:osd.1> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.1.service 2024-03-16T10:00:29.032 INFO:tasks.cephadm:Deploying osd.2 on smithi194 with /dev/vg_nvme/lv_4... 2024-03-16T10:00:29.032 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T10:00:29.275 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:29.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:29 smithi105 ceph-mon[36559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:29.646 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:29 smithi194 ceph-mon[36445]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:29 smithi039 ceph-mon[31510]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:00:30.643 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:30 smithi194 ceph-mon[36445]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:00:30.643 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:30 smithi194 ceph-mon[36445]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:30.643 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:30 smithi194 ceph-mon[36445]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T10:00:30.643 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:30 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:30.644 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:30 smithi194 ceph-mon[36445]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:30.680 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-mon[36559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:00:30.680 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-mon[36559]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:30.680 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-mon[36559]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T10:00:30.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:30.681 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-mon[36559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:30.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:30 smithi039 ceph-mon[31510]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:00:30.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:30 smithi039 ceph-mon[31510]: from='osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:30.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:30 smithi039 ceph-mon[31510]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T10:00:30.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:30.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:30 smithi039 ceph-mon[31510]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112683 s, 931 MB/s 2024-03-16T10:00:31.083 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T10:00:31.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1[41146]: 2024-03-16T10:00:30.861+0000 7fe0f2249700 -1 osd.1 0 waiting for initial osdmap 2024-03-16T10:00:31.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1[41146]: 2024-03-16T10:00:30.864+0000 7fe0ea034700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:00:31.615 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: from='osd.1 ' entity='osd.1' 2024-03-16T10:00:31.667 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: from='osd.1 ' entity='osd.1' 2024-03-16T10:00:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:00:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T10:00:31.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: from='osd.1 ' entity='osd.1' 2024-03-16T10:00:31.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:31.849 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] boot 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:32.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:32.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:32.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:32 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:00:32.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:00:32.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] boot 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:32.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:32 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: osd.1 [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] boot 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:32.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:32 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:33.234 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.230+0000 7f3488f58700 1 -- 172.21.15.194:0/3642157568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484101200 msgr2=0x7f3484101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:33.234 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.230+0000 7f3488f58700 1 --2- 172.21.15.194:0/3642157568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484101200 0x7f3484101620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3474009990 tx=0x7f3474009ca0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.234 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 -- 172.21.15.194:0/3642157568 shutdown_connections 2024-03-16T10:00:33.234 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 --2- 172.21.15.194:0/3642157568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3484102da0 0x7f3484109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 --2- 172.21.15.194:0/3642157568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484101200 0x7f3484101620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 --2- 172.21.15.194:0/3642157568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 0x7f3484102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 -- 172.21.15.194:0/3642157568 >> 172.21.15.194:0/3642157568 conn(0x7f34840fc7c0 msgr2=0x7f34840febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 -- 172.21.15.194:0/3642157568 shutdown_connections 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 -- 172.21.15.194:0/3642157568 wait complete. 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 Processor -- start 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.231+0000 7f3488f58700 1 -- start start 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3484101200 0x7f34841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 0x7f3484198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f348410ce40 con 0x7f3484101200 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f348410ccc0 con 0x7f3484102400 2024-03-16T10:00:33.235 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f348410cfc0 con 0x7f3484102da0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:40640/0 (socket says 172.21.15.194:40640) 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 -- 172.21.15.194:0/708622390 learned_addr learned my addr 172.21.15.194:0/708622390 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3481d9b700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 0x7f3484198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 -- 172.21.15.194:0/708622390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 msgr2=0x7f3484198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 0x7f3484198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 -- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3484101200 msgr2=0x7f34841985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3484101200 0x7f34841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 -- 172.21.15.194:0/708622390 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34841a0240 con 0x7f3484102da0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3482d9d700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f347c00b9a0 tx=0x7f347c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c019070 con 0x7f3484102da0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f347c00ce10 con 0x7f3484102da0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34841a0530 con 0x7f3484102da0 2024-03-16T10:00:33.236 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c0073d0 con 0x7f3484102da0 2024-03-16T10:00:33.237 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.232+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34841a0af0 con 0x7f3484102da0 2024-03-16T10:00:33.237 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.233+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3484068f90 con 0x7f3484102da0 2024-03-16T10:00:33.238 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.234+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f347c01c070 con 0x7f3484102da0 2024-03-16T10:00:33.238 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.235+0000 7f34737fe700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 0x7f346c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:33.238 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.235+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f347c0cc520 con 0x7f3484102da0 2024-03-16T10:00:33.238 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.235+0000 7f348259c700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 0x7f346c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:33.238 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.236+0000 7f348259c700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 0x7f346c0a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3474005f50 tx=0x7f3474005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:33.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.240+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f347c09ae10 con 0x7f3484102da0 2024-03-16T10:00:33.445 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:33.444+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f34841a0db0 con 0x7f346c0a2800 2024-03-16T10:00:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:33 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi105 2024-03-16T10:00:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:33 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-16T10:00:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:33 smithi194 ceph-mon[36445]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:33 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi105 2024-03-16T10:00:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:33 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-16T10:00:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:33 smithi039 ceph-mon[31510]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:33 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi105 2024-03-16T10:00:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:33 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-16T10:00:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:33 smithi105 ceph-mon[36559]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:00:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:34 smithi194 ceph-mon[36445]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:34 smithi194 ceph-mon[36445]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T10:00:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:34 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:34 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:34.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:34 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:34 smithi039 ceph-mon[31510]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:34 smithi039 ceph-mon[31510]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T10:00:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:34 smithi105 ceph-mon[36559]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:00:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:34 smithi105 ceph-mon[36559]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T10:00:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:34 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:34 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:34 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:35 smithi039 ceph-mon[31510]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:35 smithi105 ceph-mon[36559]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:35 smithi194 ceph-mon[36445]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:36.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:36 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/4020089410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:36 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:36 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]': finished 2024-03-16T10:00:36.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:36 smithi194 ceph-mon[36445]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T10:00:36.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:36 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:36 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/4020089410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:36 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:36 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]': finished 2024-03-16T10:00:36.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:36 smithi039 ceph-mon[31510]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T10:00:36.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:36 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:36 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/4020089410' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:36 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]: dispatch 2024-03-16T10:00:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:36 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f3873f3-0b87-42c9-a34b-f62099b77f39"}]': finished 2024-03-16T10:00:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:36 smithi105 ceph-mon[36559]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T10:00:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:36 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:37 smithi039 ceph-mon[31510]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:37 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/2888308371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:37 smithi105 ceph-mon[36559]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:37 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/2888308371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:37 smithi194 ceph-mon[36445]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:37 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/2888308371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:00:39.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:39 smithi039 ceph-mon[31510]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:39 smithi105 ceph-mon[36559]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:39.876 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:39 smithi194 ceph-mon[36445]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:41.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:41 smithi194 ceph-mon[36445]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:41 smithi039 ceph-mon[31510]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:41.818 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:41 smithi105 ceph-mon[36559]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:43.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:43 smithi194 ceph-mon[36445]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:43.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:00:43.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:43.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:43 smithi194 ceph-mon[36445]: Deploying daemon osd.2 on smithi194 2024-03-16T10:00:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:43 smithi039 ceph-mon[31510]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:00:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:43 smithi039 ceph-mon[31510]: Deploying daemon osd.2 on smithi194 2024-03-16T10:00:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:43 smithi105 ceph-mon[36559]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:00:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:43 smithi105 ceph-mon[36559]: Deploying daemon osd.2 on smithi194 2024-03-16T10:00:45.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:45 smithi039 ceph-mon[31510]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:45.849 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:45 smithi194 ceph-mon[36445]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:45.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:45 smithi105 ceph-mon[36559]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:47 smithi039 ceph-mon[31510]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:47.850 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:47 smithi194 ceph-mon[36445]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:47 smithi105 ceph-mon[36559]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:50.022 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:50 smithi194 ceph-mon[36445]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:50.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:50 smithi105 ceph-mon[36559]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:50.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:50 smithi039 ceph-mon[31510]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:51.180 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.178+0000 7f34737fe700 1 -- 172.21.15.194:0/708622390 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34841a0db0 con 0x7f346c0a2800 2024-03-16T10:00:51.180 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 2 on host 'smithi194' 2024-03-16T10:00:51.181 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 msgr2=0x7f346c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:51.181 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 0x7f346c0a4cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3474005f50 tx=0x7f3474005e80 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 msgr2=0x7f348419fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f347c00b9a0 tx=0x7f347c00bcb0 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 shutdown_connections 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f346c0a2800 0x7f346c0a4cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3484102da0 0x7f348419fba0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3484101200 0x7f34841985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 --2- 172.21.15.194:0/708622390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3484102400 0x7f3484198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 >> 172.21.15.194:0/708622390 conn(0x7f34840fc7c0 msgr2=0x7f34840fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 shutdown_connections 2024-03-16T10:00:51.182 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:00:51.180+0000 7f3488f58700 1 -- 172.21.15.194:0/708622390 wait complete. 2024-03-16T10:00:51.794 DEBUG:teuthology.orchestra.run.smithi194:osd.2> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.2.service 2024-03-16T10:00:51.796 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2024-03-16T10:00:51.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T10:00:52.087 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:00:51.890+0000 7fdb729787c0 -1 osd.2 0 log_to_monitors true 2024-03-16T10:00:52.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:52.087 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:52.088 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:00:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:53.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:52 smithi039 ceph-mon[31510]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:53.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:52 smithi039 ceph-mon[31510]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:52 smithi105 ceph-mon[36559]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:52 smithi105 ceph-mon[36559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:53.194 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:52 smithi194 ceph-mon[36445]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:53.195 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:52 smithi194 ceph-mon[36445]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:00:54.020 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:54.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:00:54.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T10:00:54.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:54.021 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:53 smithi039 ceph-mon[31510]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:53 smithi105 ceph-mon[36559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:00:53.851+0000 7fdb67fdd700 -1 osd.2 0 waiting for initial osdmap 2024-03-16T10:00:54.207 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:00:53.858+0000 7fdb63dd0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:00:54.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:54.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:00:54.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: from='osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T10:00:54.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:54.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:53 smithi194 ceph-mon[36445]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:00:54.967 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T10:00:54.967 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T10:00:54.967 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-03-16T10:00:54.968 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-03-16T10:00:54.968 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0315905 s, 332 MB/s 2024-03-16T10:00:54.968 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-03-16T10:00:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:00:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T10:00:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:00:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:00:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:55.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-03-16T10:00:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:00:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:00:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:56.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi194 2024-03-16T10:00:56.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T10:00:56.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] boot 2024-03-16T10:00:56.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T10:00:56.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi194 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T10:00:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] boot 2024-03-16T10:00:56.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T10:00:56.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi194 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: osd.2 [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] boot 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T10:00:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:00:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:56 smithi039 ceph-mon[31510]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T10:00:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:57.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:56 smithi194 ceph-mon[36445]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T10:00:57.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:56 smithi105 ceph-mon[36559]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T10:00:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:58.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:57 smithi039 ceph-mon[31510]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:00:58.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:58.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:57 smithi039 ceph-mon[31510]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T10:00:58.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:57 smithi194 ceph-mon[36445]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:00:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:57 smithi194 ceph-mon[36445]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T10:00:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 -- 172.21.15.39:0/4123117943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8103510 msgr2=0x7f47f8103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 --2- 172.21.15.39:0/4123117943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8103510 0x7f47f8103970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f47e40099c0 tx=0x7f47e4009cd0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 -- 172.21.15.39:0/4123117943 shutdown_connections 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 --2- 172.21.15.39:0/4123117943 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f810a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 --2- 172.21.15.39:0/4123117943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8102310 0x7f47f8102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 --2- 172.21.15.39:0/4123117943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8103510 0x7f47f8103970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 -- 172.21.15.39:0/4123117943 >> 172.21.15.39:0/4123117943 conn(0x7f47f80fd870 msgr2=0x7f47f80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 -- 172.21.15.39:0/4123117943 shutdown_connections 2024-03-16T10:00:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.356+0000 7f47fd200700 1 -- 172.21.15.39:0/4123117943 wait complete. 2024-03-16T10:00:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.357+0000 7f47fd200700 1 Processor -- start 2024-03-16T10:00:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.357+0000 7f47fd200700 1 -- start start 2024-03-16T10:00:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 0x7f47f819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8103510 0x7f47f819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810df00 con 0x7f47f8102310 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810dd80 con 0x7f47f8103510 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47fd200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f47f810e080 con 0x7f47f8103eb0 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47f5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 0x7f47f819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47f5e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 0x7f47f819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45090/0 (socket says 172.21.15.39:45090) 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47f5e00700 1 -- 172.21.15.39:0/3225137509 learned_addr learned my addr 172.21.15.39:0/3225137509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47f5400700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8103510 0x7f47f819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.358+0000 7f47f6800700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 -- 172.21.15.39:0/3225137509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8103510 msgr2=0x7f47f819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8103510 0x7f47f819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 -- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 msgr2=0x7f47f819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 0x7f47f819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:00:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 -- 172.21.15.39:0/3225137509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f81a2380 con 0x7f47f8103eb0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47f6800700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f81a1ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009f60 tx=0x7f47f0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f00092e0 con 0x7f47f8103eb0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f0004bb0 con 0x7f47f8103eb0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f81a2610 con 0x7f47f8103eb0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.359+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0004280 con 0x7f47f8103eb0 2024-03-16T10:00:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.360+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f81a2b50 con 0x7f47f8103eb0 2024-03-16T10:00:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.361+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f8068f90 con 0x7f47f8103eb0 2024-03-16T10:00:58.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:57 smithi105 ceph-mon[36559]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:00:58.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:57 smithi105 ceph-mon[36559]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T10:00:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:00:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.364+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f47f001b030 con 0x7f47f8103eb0 2024-03-16T10:00:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.366+0000 7f47eea00700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 0x7f47dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:00:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.366+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f47f00cd020 con 0x7f47f8103eb0 2024-03-16T10:00:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.366+0000 7f47f5e00700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 0x7f47dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:00:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.367+0000 7f47f5e00700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 0x7f47dc0a4d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009680 tx=0x7f47e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:00:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.370+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f47f0022020 con 0x7f47f8103eb0 2024-03-16T10:00:58.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.611+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f47f8075c20 con 0x7f47dc0a2880 2024-03-16T10:00:58.923 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:58 smithi039 sudo[50999]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:00:58.923 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:58 smithi039 sudo[50999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:00:58.924 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:58 smithi039 sudo[50999]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:00:58.924 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:58 smithi039 sudo[50999]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:00:58.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:00:58.932+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f47f0012050 con 0x7f47f8103eb0 2024-03-16T10:00:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:58 smithi194 ceph-mon[36445]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T10:00:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:59.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:59.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:58 smithi039 ceph-mon[31510]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T10:00:59.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:59.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:59.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:00:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:58 smithi105 ceph-mon[36559]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T10:00:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:00:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:00:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:00:59.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:59 smithi039 sudo[51068]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:00:59.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:59 smithi039 sudo[51068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:00:59.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:59 smithi039 sudo[51068]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:00:59.506 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:00:59 smithi039 sudo[51068]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:00:59.764 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:00:59.764 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:00:59.764 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:00:59.764 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45003]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:00.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:01:00.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:00.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:00.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:00:59 smithi105 sudo[45006]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:00:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45529]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:01:00.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:00.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45529]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:00.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45529]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:00.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:00.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:00.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T10:01:00.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T10:01:00.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:00:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:00:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:00.706 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45532]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:01:00.706 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45532]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:00.706 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45532]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:00.706 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:01:00 smithi194 sudo[45532]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:00 smithi039 sudo[51165]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:01:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:00 smithi039 sudo[51165]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:00 smithi039 sudo[51165]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:00 smithi039 sudo[51165]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:01.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:01 smithi039 sudo[51192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:01:01.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:01 smithi039 sudo[51192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:01.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:01 smithi039 sudo[51192]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:01.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:01 smithi039 sudo[51192]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:02.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:01 smithi194 sudo[45550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:01:02.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:01 smithi194 sudo[45550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:02.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:01 smithi194 sudo[45550]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:02.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:01 smithi194 sudo[45550]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:02.257 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:02.257 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 sudo[45553]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 sudo[45553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 sudo[45553]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 sudo[45553]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:02 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:02.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:02 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:02.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:02 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:02.520 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:01:02.520 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:02.520 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:02.520 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45024]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:02.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45028]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:01:02.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:01:02.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45028]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:01:02.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:02 smithi105 sudo[45028]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2722550265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]': finished 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T10:01:03.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:03.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.404 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2722550265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]': finished 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T10:01:03.405 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2722550265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]: dispatch 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6455fd4-f277-4db1-a5c8-0fb262d3bb49"}]': finished 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T10:01:03.458 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:04.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:04 smithi105 ceph-mon[36559]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/581859749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:04.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:04 smithi039 ceph-mon[31510]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:04.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/581859749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:04 smithi194 ceph-mon[36445]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/581859749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:06.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:06 smithi105 ceph-mon[36559]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:06.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:06 smithi194 ceph-mon[36445]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:06 smithi039 ceph-mon[31510]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:07.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:07 smithi105 ceph-mon[36559]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:07.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:07 smithi039 ceph-mon[31510]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:07.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:07 smithi194 ceph-mon[36445]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:09.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:09 smithi194 ceph-mon[36445]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2024-03-16T10:01:09.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:09 smithi039 ceph-mon[31510]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2024-03-16T10:01:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:09 smithi105 ceph-mon[36559]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2024-03-16T10:01:11.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:11 smithi039 ceph-mon[31510]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-03-16T10:01:11.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:11 smithi194 ceph-mon[36445]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-03-16T10:01:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:11 smithi105 ceph-mon[36559]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2024-03-16T10:01:12.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:01:12.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:12.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:12 smithi194 ceph-mon[36445]: Deploying daemon osd.3 on smithi039 2024-03-16T10:01:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:01:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:12.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:12 smithi039 ceph-mon[31510]: Deploying daemon osd.3 on smithi039 2024-03-16T10:01:12.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:01:12.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:12.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:12 smithi105 ceph-mon[36559]: Deploying daemon osd.3 on smithi039 2024-03-16T10:01:13.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:13 smithi039 ceph-mon[31510]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-03-16T10:01:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:13 smithi105 ceph-mon[36559]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-03-16T10:01:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:13 smithi194 ceph-mon[36445]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2024-03-16T10:01:15.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:15 smithi194 ceph-mon[36445]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 906 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-16T10:01:15.476 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:15 smithi039 ceph-mon[31510]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 906 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-16T10:01:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:15 smithi105 ceph-mon[36559]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 906 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-16T10:01:17.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:17 smithi039 ceph-mon[31510]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:17 smithi194 ceph-mon[36445]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:17 smithi105 ceph-mon[36559]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:19.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:19 smithi039 ceph-mon[31510]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:19 smithi194 ceph-mon[36445]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:19 smithi105 ceph-mon[36559]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-16T10:01:21.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:21 smithi039 ceph-mon[31510]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:21.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.576 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:21 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:21 smithi105 ceph-mon[36559]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:21 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:21 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:21 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:21 smithi194 ceph-mon[36445]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:21.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:21 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:23.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:23 smithi039 ceph-mon[31510]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:23 smithi105 ceph-mon[36559]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:23 smithi194 ceph-mon[36445]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.053+0000 7f47eea00700 1 -- 172.21.15.39:0/3225137509 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f47f8075c20 con 0x7f47dc0a2880 2024-03-16T10:01:25.056 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2024-03-16T10:01:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 msgr2=0x7f47dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 0x7f47dc0a4d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009680 tx=0x7f47e8009260 comp rx=0 tx=0).stop 2024-03-16T10:01:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 msgr2=0x7f47f81a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f81a1ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009f60 tx=0x7f47f0009f90 comp rx=0 tx=0).stop 2024-03-16T10:01:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 shutdown_connections 2024-03-16T10:01:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f47dc0a2880 0x7f47dc0a4d40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47f8103eb0 0x7f47f81a1ce0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8102310 0x7f47f819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 --2- 172.21.15.39:0/3225137509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f8103510 0x7f47f819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.057+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 >> 172.21.15.39:0/3225137509 conn(0x7f47f80fd870 msgr2=0x7f47f8074500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.058+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 shutdown_connections 2024-03-16T10:01:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:01:25.058+0000 7f47fd200700 1 -- 172.21.15.39:0/3225137509 wait complete. 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:24 smithi105 ceph-mon[36559]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:24 smithi194 ceph-mon[36445]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:25.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:25.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:25.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:24 smithi039 ceph-mon[31510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:01:25.912 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.3.service 2024-03-16T10:01:25.916 INFO:tasks.cephadm:Deploying osd.4 on smithi105 with /dev/vg_nvme/lv_3... 2024-03-16T10:01:25.917 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T10:01:26.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:26.160 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:01:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:01:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T10:01:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:26.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:26.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:26.282 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:26.283 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] boot 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T10:01:26.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:01:27.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T10:01:27.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] boot 2024-03-16T10:01:27.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T10:01:27.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: osd.3 [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] boot 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T10:01:27.246 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112757 s, 930 MB/s 2024-03-16T10:01:28.112 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-16T10:01:28.136 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:27 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:01:28.136 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:27 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:01:28.136 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:27 smithi105 ceph-mon[36559]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:28.136 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:27 smithi105 ceph-mon[36559]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T10:01:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:27 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:01:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:27 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:01:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:27 smithi194 ceph-mon[36445]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:27 smithi194 ceph-mon[36445]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T10:01:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:27 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:01:28.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:27 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:01:28.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:27 smithi039 ceph-mon[31510]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:01:28.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:27 smithi039 ceph-mon[31510]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T10:01:28.806 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-03-16T10:01:29.048 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:01:29.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:29 smithi039 ceph-mon[31510]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:29.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:29 smithi039 ceph-mon[31510]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T10:01:29.843 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:29 smithi105 ceph-mon[36559]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:29.843 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:29 smithi105 ceph-mon[36559]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T10:01:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:29 smithi194 ceph-mon[36445]: pgmap v115: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:29 smithi194 ceph-mon[36445]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T10:01:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 -- 172.21.15.105:0/2964144585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c102dc0 msgr2=0x7f546c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 --2- 172.21.15.105:0/2964144585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c102dc0 0x7f546c1051b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5460009990 tx=0x7f5460009ca0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 -- 172.21.15.105:0/2964144585 shutdown_connections 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 --2- 172.21.15.105:0/2964144585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c108020 0x7f546c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 --2- 172.21.15.105:0/2964144585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c102dc0 0x7f546c1051b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 --2- 172.21.15.105:0/2964144585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f546c1056f0 0x7f546c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 -- 172.21.15.105:0/2964144585 >> 172.21.15.105:0/2964144585 conn(0x7f546c0fc7f0 msgr2=0x7f546c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 -- 172.21.15.105:0/2964144585 shutdown_connections 2024-03-16T10:01:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.616+0000 7f54720bb700 1 -- 172.21.15.105:0/2964144585 wait complete. 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 Processor -- start 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 -- start start 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 0x7f546c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f546c108020 0x7f546c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10df80 con 0x7f546c108020 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10de00 con 0x7f546c1056f0 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f54720bb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f546c10e100 con 0x7f546c102dc0 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f546affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f546affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38278/0 (socket says 172.21.15.105:38278) 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.617+0000 7f546affd700 1 -- 172.21.15.105:0/1916308754 learned_addr learned my addr 172.21.15.105:0/1916308754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-16T10:01:30.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546b7fe700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 0x7f546c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 -- 172.21.15.105:0/1916308754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 msgr2=0x7f546c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 0x7f546c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 -- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f546c108020 msgr2=0x7f546c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f546c108020 0x7f546c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 -- 172.21.15.105:0/1916308754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c1a01f0 con 0x7f546c1056f0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546b7fe700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 0x7f546c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f546affd700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b530 tx=0x7f545c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c015020 con 0x7f546c1056f0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1a04e0 con 0x7f546c1056f0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c011d10 con 0x7f546c1056f0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c00f2e0 con 0x7f546c1056f0 2024-03-16T10:01:30.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.618+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1a0af0 con 0x7f546c1056f0 2024-03-16T10:01:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.619+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c068f90 con 0x7f546c1056f0 2024-03-16T10:01:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.620+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f545c018020 con 0x7f546c1056f0 2024-03-16T10:01:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.620+0000 7f5468ff9700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 0x7f54540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:30.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.621+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f545c0cd550 con 0x7f546c1056f0 2024-03-16T10:01:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.621+0000 7f546b7fe700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 0x7f54540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:30.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.622+0000 7f546b7fe700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 0x7f54540a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54600039c0 tx=0x7f5460005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:01:30.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.624+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f545c01f020 con 0x7f546c1056f0 2024-03-16T10:01:30.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:30.831+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f546c062710 con 0x7f54540a2890 2024-03-16T10:01:30.833 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:30 smithi105 ceph-mon[36559]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T10:01:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:30 smithi194 ceph-mon[36445]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T10:01:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:30 smithi039 ceph-mon[31510]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi039 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 8545M 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.708 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:31.709 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:31 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T10:01:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi039 2024-03-16T10:01:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 8545M 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:31.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:31 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi039 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 8545M 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:32.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:32.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:31 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/2928790752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]': finished 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T10:01:33.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:33.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:33.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/2928790752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:33.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:33.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]': finished 2024-03-16T10:01:33.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T10:01:33.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/2928790752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]: dispatch 2024-03-16T10:01:34.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d166e33-4a4b-48ce-b293-7d6fe30b2adb"}]': finished 2024-03-16T10:01:34.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T10:01:34.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:34.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:34 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/811928251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:34.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:34 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/811928251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:35.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:34 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/811928251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:35 smithi105 ceph-mon[36559]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:01:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:35 smithi194 ceph-mon[36445]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:01:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:35 smithi039 ceph-mon[31510]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:01:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:37 smithi039 ceph-mon[31510]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:01:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:37 smithi105 ceph-mon[36559]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:01:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:37 smithi194 ceph-mon[36445]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:01:39.766 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:39 smithi105 ceph-mon[36559]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:01:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:39 smithi194 ceph-mon[36445]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:01:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:39 smithi039 ceph-mon[31510]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:01:41.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:41 smithi105 ceph-mon[36559]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:01:41.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:01:41.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:41.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:41 smithi105 ceph-mon[36559]: Deploying daemon osd.4 on smithi105 2024-03-16T10:01:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:41 smithi194 ceph-mon[36445]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:01:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:01:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:41 smithi194 ceph-mon[36445]: Deploying daemon osd.4 on smithi105 2024-03-16T10:01:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:41 smithi039 ceph-mon[31510]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:01:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:01:42.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:42.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:41 smithi039 ceph-mon[31510]: Deploying daemon osd.4 on smithi105 2024-03-16T10:01:43.863 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:43 smithi105 ceph-mon[36559]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:01:43.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:43 smithi194 ceph-mon[36445]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:01:44.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:43 smithi039 ceph-mon[31510]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:01:45.828 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:45 smithi105 ceph-mon[36559]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:45.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:45 smithi194 ceph-mon[36445]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:45 smithi039 ceph-mon[31510]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:47 smithi194 ceph-mon[36445]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:47 smithi039 ceph-mon[31510]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:47 smithi105 ceph-mon[36559]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:49 smithi194 ceph-mon[36445]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:49 smithi039 ceph-mon[31510]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:49.768 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:49 smithi105 ceph-mon[36559]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:49.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.871+0000 7f5468ff9700 1 -- 172.21.15.105:0/1916308754 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f546c062710 con 0x7f54540a2890 2024-03-16T10:01:49.872 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 4 on host 'smithi105' 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 msgr2=0x7f54540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 0x7f54540a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54600039c0 tx=0x7f5460005fb0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 msgr2=0x7f546c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f545c00b530 tx=0x7f545c00b8f0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 shutdown_connections 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54540a2890 0x7f54540a4d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f546c108020 0x7f546c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c102dc0 0x7f546c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 --2- 172.21.15.105:0/1916308754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f546c1056f0 0x7f546c198ad0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 >> 172.21.15.105:0/1916308754 conn(0x7f546c0fc7f0 msgr2=0x7f546c103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 shutdown_connections 2024-03-16T10:01:49.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:01:49.873+0000 7f54720bb700 1 -- 172.21.15.105:0/1916308754 wait complete. 2024-03-16T10:01:50.489 DEBUG:teuthology.orchestra.run.smithi105:osd.4> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.4.service 2024-03-16T10:01:50.492 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_3... 2024-03-16T10:01:50.492 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T10:01:50.738 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.111 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:01:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:52.093 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:51 smithi194 ceph-mon[36445]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-mon[36559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.115 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:01:51.838+0000 7fa7b14d0700 -1 osd.4 0 waiting for initial osdmap 2024-03-16T10:01:52.115 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:01:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:01:51.843+0000 7fa7ad2c3700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:01:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:01:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: from='osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T10:01:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:51 smithi039 ceph-mon[31510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:01:52.785 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T10:01:52.785 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T10:01:52.785 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T10:01:52.786 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T10:01:52.786 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136109 s, 770 MB/s 2024-03-16T10:01:52.786 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T10:01:53.106 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:52 smithi194 ceph-mon[36445]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:01:53.106 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:52 smithi194 ceph-mon[36445]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T10:01:53.106 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:52 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.106 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:52 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:52 smithi105 ceph-mon[36559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:01:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:52 smithi105 ceph-mon[36559]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T10:01:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:52 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:52 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:52 smithi039 ceph-mon[31510]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:01:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:52 smithi039 ceph-mon[31510]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T10:01:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:52 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:52 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.417 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-03-16T10:01:53.658 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:01:53.920 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:01:53.920 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:01:53.920 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:53.921 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] boot 2024-03-16T10:01:53.921 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T10:01:53.921 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] boot 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T10:01:53.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: osd.4 [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] boot 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T10:01:54.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi105 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:54.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:54.909 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:54.909 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:54.909 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:54.909 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3185460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0101200 msgr2=0x7f6fd0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:55.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3185460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0101200 0x7f6fd0101620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009990 tx=0x7f6fc0009ca0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3185460998 shutdown_connections 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3185460998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fd0102da0 0x7f6fd0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3185460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0101200 0x7f6fd0101620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3185460998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 0x7f6fd0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3185460998 >> 172.21.15.194:0/3185460998 conn(0x7f6fd00fc7a0 msgr2=0x7f6fd00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3185460998 shutdown_connections 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3185460998 wait complete. 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.242+0000 7f6fd66a5700 1 Processor -- start 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 -- start start 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fd0101200 0x7f6fd0105140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 0x7f6fd01037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:55.246 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd010ce70 con 0x7f6fd0101200 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd010ccf0 con 0x7f6fd0102400 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd66a5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd010cff0 con 0x7f6fd0102da0 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:35676/0 (socket says 172.21.15.194:35676) 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 -- 172.21.15.194:0/3578219358 learned_addr learned my addr 172.21.15.194:0/3578219358 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fcf7fe700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 0x7f6fd01037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 -- 172.21.15.194:0/3578219358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 msgr2=0x7f6fd01037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 0x7f6fd01037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 -- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fd0101200 msgr2=0x7f6fd0105140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:01:55.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fd0101200 0x7f6fd0105140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 -- 172.21.15.194:0/3578219358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd01045f0 con 0x7f6fd0102da0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.243+0000 7f6fd4c42700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b970 tx=0x7f6fc800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.244+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8019070 con 0x7f6fd0102da0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.244+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd01a2740 con 0x7f6fd0102da0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.244+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc800ce10 con 0x7f6fd0102da0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.244+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8007370 con 0x7f6fd0102da0 2024-03-16T10:01:55.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.244+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd01a2c80 con 0x7f6fd0102da0 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.245+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd0068f90 con 0x7f6fd0102da0 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.246+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f6fc801c070 con 0x7f6fd0102da0 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.246+0000 7f6fcd7fa700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 0x7f6fb80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.246+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f6fc80cce10 con 0x7f6fd0102da0 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.247+0000 7f6fcffff700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 0x7f6fb80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:01:55.249 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.248+0000 7f6fcffff700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 0x7f6fb80a4d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00098e0 tx=0x7f6fc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:01:55.252 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.251+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6fc809af20 con 0x7f6fd0102da0 2024-03-16T10:01:55.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T10:01:55.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi105 2024-03-16T10:01:55.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:55.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi105 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:01:55.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:01:55.458 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:01:55.455+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6fd00fe990 con 0x7f6fb80a2890 2024-03-16T10:01:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:55 smithi194 ceph-mon[36445]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:55 smithi039 ceph-mon[31510]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:55 smithi105 ceph-mon[36559]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:01:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:01:56.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:01:57.171 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:56 smithi194 ceph-mon[36445]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:57.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:56 smithi039 ceph-mon[31510]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:56 smithi105 ceph-mon[36559]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:01:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:57 smithi194 ceph-mon[36445]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:57 smithi039 ceph-mon[31510]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:57 smithi105 ceph-mon[36559]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/1743899623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]': finished 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:01:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:58 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/529968591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:59.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/1743899623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]': finished 2024-03-16T10:01:59.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T10:01:59.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:01:59.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/529968591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:01:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/1743899623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]: dispatch 2024-03-16T10:01:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00"}]': finished 2024-03-16T10:01:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T10:01:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:01:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:58 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/529968591' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:01:59 smithi039 ceph-mon[31510]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:01:59 smithi105 ceph-mon[36559]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:01:59 smithi194 ceph-mon[36445]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:02.289 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:02 smithi194 ceph-mon[36445]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:02 smithi105 ceph-mon[36559]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:02.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:02 smithi039 ceph-mon[31510]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:04.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:04 smithi194 ceph-mon[36445]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:04.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:04 smithi105 ceph-mon[36559]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:04 smithi039 ceph-mon[31510]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:06.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:06 smithi194 ceph-mon[36445]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:06.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:02:06.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:06.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:06 smithi105 ceph-mon[36559]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:06.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:02:06.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:06 smithi039 ceph-mon[31510]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:02:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:07.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:07 smithi105 ceph-mon[36559]: Deploying daemon osd.5 on smithi194 2024-03-16T10:02:07.420 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:07 smithi194 ceph-mon[36445]: Deploying daemon osd.5 on smithi194 2024-03-16T10:02:07.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:07 smithi039 ceph-mon[31510]: Deploying daemon osd.5 on smithi194 2024-03-16T10:02:08.359 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:08 smithi194 ceph-mon[36445]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:08.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:08 smithi105 ceph-mon[36559]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:08.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:08 smithi039 ceph-mon[31510]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:10.146 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:10 smithi194 ceph-mon[36445]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:10.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:10 smithi105 ceph-mon[36559]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:10.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:10 smithi039 ceph-mon[31510]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:12.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:11 smithi105 ceph-mon[36559]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:12.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:11 smithi194 ceph-mon[36445]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:11 smithi039 ceph-mon[31510]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:14.072 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:14.073 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.073 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.073 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:14.073 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:14.073 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:14.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:14.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:14.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:14.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.489+0000 7f6fcd7fa700 1 -- 172.21.15.194:0/3578219358 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6fd00fe990 con 0x7f6fb80a2890 2024-03-16T10:02:14.491 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2024-03-16T10:02:14.493 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 msgr2=0x7f6fb80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:14.493 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 0x7f6fb80a4d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00098e0 tx=0x7f6fc0005fb0 comp rx=0 tx=0).stop 2024-03-16T10:02:14.493 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 msgr2=0x7f6fd0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:14.493 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800b970 tx=0x7f6fc800bc80 comp rx=0 tx=0).stop 2024-03-16T10:02:14.493 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 shutdown_connections 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6fb80a2890 0x7f6fb80a4d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fd0102da0 0x7f6fd0103d20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fd0101200 0x7f6fd0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 --2- 172.21.15.194:0/3578219358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fd0102400 0x7f6fd01037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 >> 172.21.15.194:0/3578219358 conn(0x7f6fd00fc7a0 msgr2=0x7f6fd00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 shutdown_connections 2024-03-16T10:02:14.494 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:02:14.492+0000 7f6fd66a5700 1 -- 172.21.15.194:0/3578219358 wait complete. 2024-03-16T10:02:15.076 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.5.service 2024-03-16T10:02:15.078 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2024-03-16T10:02:15.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T10:02:15.537 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:15 smithi194 ceph-mon[36445]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:15.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:15 smithi039 ceph-mon[31510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:15 smithi105 ceph-mon[36559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:02:16.729 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:16 smithi039 ceph-mon[31510]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:02:16.729 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:16 smithi039 ceph-mon[31510]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T10:02:16.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:16 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:16.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:16 smithi039 ceph-mon[31510]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:16 smithi039 ceph-mon[31510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:16 smithi105 ceph-mon[36559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:02:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:16 smithi105 ceph-mon[36559]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T10:02:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:16 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:16 smithi105 ceph-mon[36559]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:16 smithi105 ceph-mon[36559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-mon[36445]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:02:16.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-mon[36445]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T10:02:16.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:16.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-mon[36445]: from='osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-mon[36445]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:02:16.957 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:02:16.463+0000 7f5cbdcee700 -1 osd.5 0 waiting for initial osdmap 2024-03-16T10:02:16.957 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:02:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:02:16.472+0000 7f5cb9ae1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:02:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] boot 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T10:02:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:17 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:17.664 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] boot 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T10:02:17.665 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:17 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: osd.5 [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] boot 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T10:02:17.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:17 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:02:18.460 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T10:02:18.460 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T10:02:18.460 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-03-16T10:02:18.460 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-03-16T10:02:18.461 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0372949 s, 281 MB/s 2024-03-16T10:02:18.461 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-03-16T10:02:18.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi194 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:18.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:18 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi194 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:18.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:18.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:18 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:02:18.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:02:18.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T10:02:18.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi194 2024-03-16T10:02:18.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:18.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:18 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:19.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-03-16T10:02:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:19 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T10:02:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:19 smithi039 ceph-mon[31510]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:19 smithi039 ceph-mon[31510]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T10:02:19.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:19 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T10:02:19.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:19 smithi105 ceph-mon[36559]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:19.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:19 smithi105 ceph-mon[36559]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T10:02:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:02:19.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:19 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T10:02:19.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:19 smithi194 ceph-mon[36445]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:19.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:19 smithi194 ceph-mon[36445]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T10:02:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:20 smithi039 ceph-mon[31510]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T10:02:20.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:20 smithi105 ceph-mon[36559]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T10:02:20.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:20 smithi194 ceph-mon[36445]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T10:02:21.810 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:21 smithi039 ceph-mon[31510]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:21.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:21 smithi039 ceph-mon[31510]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T10:02:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:21 smithi105 ceph-mon[36559]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:21 smithi105 ceph-mon[36559]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T10:02:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:21 smithi194 ceph-mon[36445]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:21 smithi194 ceph-mon[36445]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T10:02:21.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.984+0000 7fcbac000700 1 -- 172.21.15.39:0/978131967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba4107ff0 msgr2=0x7fcba410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.984+0000 7fcbac000700 1 --2- 172.21.15.39:0/978131967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba4107ff0 0x7fcba410a460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00b0a0 tx=0x7fcb9c00b3b0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 -- 172.21.15.39:0/978131967 shutdown_connections 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 --2- 172.21.15.39:0/978131967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba4107ff0 0x7fcba410a460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 --2- 172.21.15.39:0/978131967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba4105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 --2- 172.21.15.39:0/978131967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba41056c0 0x7fcba4107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 -- 172.21.15.39:0/978131967 >> 172.21.15.39:0/978131967 conn(0x7fcba40fc7a0 msgr2=0x7fcba40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 -- 172.21.15.39:0/978131967 shutdown_connections 2024-03-16T10:02:21.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.985+0000 7fcbac000700 1 -- 172.21.15.39:0/978131967 wait complete. 2024-03-16T10:02:21.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 Processor -- start 2024-03-16T10:02:21.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 -- start start 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba41056c0 0x7fcba419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 0x7fcba41a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410df00 con 0x7fcba4102d90 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410dd80 con 0x7fcba4107ff0 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.986+0000 7fcbac000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcba410e080 con 0x7fcba41056c0 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcba9600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcba9600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58162/0 (socket says 172.21.15.39:58162) 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcba9600700 1 -- 172.21.15.39:0/1755753856 learned_addr learned my addr 172.21.15.39:0/1755753856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 0x7fcba41a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcba8c00700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba41056c0 0x7fcba419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 -- 172.21.15.39:0/1755753856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba41056c0 msgr2=0x7fcba419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:21.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba41056c0 0x7fcba419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 -- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 msgr2=0x7fcba419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:21.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:21.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcbaa000700 1 -- 172.21.15.39:0/1755753856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba41a23a0 con 0x7fcba4107ff0 2024-03-16T10:02:21.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.987+0000 7fcba9600700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcbaa000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 0x7fcba41a1d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d010 tx=0x7fcb9c004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c021070 con 0x7fcba4107ff0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c007b80 con 0x7fcba4107ff0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c01d450 con 0x7fcba4107ff0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba41a2630 con 0x7fcba4107ff0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.988+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41a2b30 con 0x7fcba4107ff0 2024-03-16T10:02:21.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.989+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba4068f90 con 0x7fcba4107ff0 2024-03-16T10:02:21.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.991+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcb9c004020 con 0x7fcba4107ff0 2024-03-16T10:02:21.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.992+0000 7fcba2000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 0x7fcb8c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:21.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.992+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fcb9c00f020 con 0x7fcba4107ff0 2024-03-16T10:02:21.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.993+0000 7fcba9600700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 0x7fcb8c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:21.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.993+0000 7fcba9600700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 0x7fcb8c0a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb980104e0 tx=0x7fcb98007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:02:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:21.997+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcb9c02a020 con 0x7fcba4107ff0 2024-03-16T10:02:22.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:22.239+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcba40627d0 con 0x7fcb8c0a2910 2024-03-16T10:02:22.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:22 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:22 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:22 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:22 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:22 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:22 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:23 smithi105 ceph-mon[36559]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:23 smithi105 ceph-mon[36559]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:23.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:23 smithi039 ceph-mon[31510]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:23.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:23 smithi039 ceph-mon[31510]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:23 smithi194 ceph-mon[36445]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:23 smithi194 ceph-mon[36445]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:25 smithi105 ceph-mon[36559]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:02:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:25 smithi194 ceph-mon[36445]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:02:26.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:25 smithi039 ceph-mon[31510]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:02:26.873 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]: dispatch 2024-03-16T10:02:26.873 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]': finished 2024-03-16T10:02:26.873 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:26 smithi194 ceph-mon[36445]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T10:02:26.873 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:27.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]: dispatch 2024-03-16T10:02:27.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]': finished 2024-03-16T10:02:27.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:26 smithi039 ceph-mon[31510]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T10:02:27.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]: dispatch 2024-03-16T10:02:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4158465980' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4d56835-6eb8-4763-bc1e-da513d084bb7"}]': finished 2024-03-16T10:02:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:26 smithi105 ceph-mon[36559]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T10:02:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:27 smithi194 ceph-mon[36445]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:02:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:27 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/951740749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:27 smithi039 ceph-mon[31510]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:02:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:27 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/951740749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:28.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:27 smithi105 ceph-mon[36559]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:02:28.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:27 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/951740749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:29 smithi194 ceph-mon[36445]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:02:30.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:29 smithi039 ceph-mon[31510]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:02:30.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:29 smithi105 ceph-mon[36559]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:02:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:31 smithi039 ceph-mon[31510]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:02:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:31 smithi194 ceph-mon[36445]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:02:32.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:31 smithi105 ceph-mon[36559]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-16T10:02:33.917 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:33 smithi105 ceph-mon[36559]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:02:33.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:33 smithi194 ceph-mon[36445]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:02:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:33 smithi039 ceph-mon[31510]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:02:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:35 smithi194 ceph-mon[36445]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:35 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T10:02:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:35 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:35 smithi039 ceph-mon[31510]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:35 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T10:02:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:35 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:36.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:35 smithi105 ceph-mon[36559]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:36.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:35 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T10:02:36.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:35 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:36.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:36 smithi194 ceph-mon[36445]: Deploying daemon osd.6 on smithi039 2024-03-16T10:02:37.000 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:36 smithi039 ceph-mon[31510]: Deploying daemon osd.6 on smithi039 2024-03-16T10:02:37.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:36 smithi105 ceph-mon[36559]: Deploying daemon osd.6 on smithi039 2024-03-16T10:02:37.709 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:37 smithi039 ceph-mon[31510]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:37 smithi194 ceph-mon[36445]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:38.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:37 smithi105 ceph-mon[36559]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:39 smithi194 ceph-mon[36445]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:39 smithi039 ceph-mon[31510]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:40.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:39 smithi105 ceph-mon[36559]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:41.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:41 smithi039 ceph-mon[31510]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:41 smithi194 ceph-mon[36445]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:41 smithi105 ceph-mon[36559]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:43.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:43 smithi194 ceph-mon[36445]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:43.981 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:43 smithi039 ceph-mon[31510]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:44.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:43 smithi105 ceph-mon[36559]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:45 smithi194 ceph-mon[36445]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:45.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:45 smithi039 ceph-mon[31510]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:45.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.485 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:45 smithi105 ceph-mon[36559]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:47 smithi194 ceph-mon[36445]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:47 smithi039 ceph-mon[31510]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:47 smithi105 ceph-mon[36559]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:49.196 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2024-03-16T10:02:49.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.195+0000 7fcba2000700 1 -- 172.21.15.39:0/1755753856 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcba40627d0 con 0x7fcb8c0a2910 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 msgr2=0x7fcb8c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 0x7fcb8c0a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb980104e0 tx=0x7fcb98007480 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 msgr2=0x7fcba41a1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 0x7fcba41a1d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00d010 tx=0x7fcb9c004dc0 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 shutdown_connections 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcb8c0a2910 0x7fcb8c0a4dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcba4107ff0 0x7fcba41a1d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcba4102d90 0x7fcba419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 --2- 172.21.15.39:0/1755753856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcba41056c0 0x7fcba419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 >> 172.21.15.39:0/1755753856 conn(0x7fcba40fc7a0 msgr2=0x7fcba41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 shutdown_connections 2024-03-16T10:02:49.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:02:49.198+0000 7fcbac000700 1 -- 172.21.15.39:0/1755753856 wait complete. 2024-03-16T10:02:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:49 smithi105 ceph-mon[36559]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:49 smithi105 ceph-mon[36559]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:49 smithi105 ceph-mon[36559]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:49 smithi194 ceph-mon[36445]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:49 smithi194 ceph-mon[36445]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:49 smithi194 ceph-mon[36445]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:49 smithi039 ceph-mon[31510]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:49 smithi039 ceph-mon[31510]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:49 smithi039 ceph-mon[31510]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T10:02:49.973 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.6.service 2024-03-16T10:02:49.978 INFO:tasks.cephadm:Deploying osd.7 on smithi105 with /dev/vg_nvme/lv_2... 2024-03-16T10:02:49.978 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T10:02:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:50.343 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T10:02:50.343 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T10:02:50.343 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:50.344 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.344 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.344 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:50.344 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:50 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:50.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:50 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:51.007 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:02:50.669+0000 7fbbe0a00700 -1 osd.6 0 waiting for initial osdmap 2024-03-16T10:02:51.007 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:02:50 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:02:50.695+0000 7fbbd5e00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:02:51.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:02:51.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T10:02:51.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:51.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:51.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: from='osd.6 ' entity='osd.6' 2024-03-16T10:02:51.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: from='osd.6 ' entity='osd.6' 2024-03-16T10:02:51.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:51.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:02:51.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: osdmap e46: 7 total, 6 up, 7 in 2024-03-16T10:02:51.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:51.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:02:51.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: from='osd.6 ' entity='osd.6' 2024-03-16T10:02:51.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113495 s, 924 MB/s 2024-03-16T10:02:52.217 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-16T10:02:52.395 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:52 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:02:52.396 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:52 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:02:52.396 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:52 smithi039 ceph-mon[31510]: osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] boot 2024-03-16T10:02:52.396 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:52 smithi039 ceph-mon[31510]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T10:02:52.396 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:52 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:52.397 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:52 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:02:52.397 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:52 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:02:52.397 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:52 smithi105 ceph-mon[36559]: osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] boot 2024-03-16T10:02:52.397 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:52 smithi105 ceph-mon[36559]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T10:02:52.397 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:52 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:52.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:52 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:02:52.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:52 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:02:52.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:52 smithi194 ceph-mon[36445]: osd.6 [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] boot 2024-03-16T10:02:52.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:52 smithi194 ceph-mon[36445]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T10:02:52.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:52 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:02:52.841 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-03-16T10:02:53.081 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:02:53.434 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:53 smithi105 ceph-mon[36559]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:53.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:53 smithi194 ceph-mon[36445]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:53 smithi039 ceph-mon[31510]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:54.171 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:54 smithi105 ceph-mon[36559]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T10:02:54.410 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:54 smithi039 ceph-mon[31510]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T10:02:54.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:54 smithi194 ceph-mon[36445]: osdmap e48: 7 total, 7 up, 7 in 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 -- 172.21.15.105:0/3925478965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 msgr2=0x7f51fc107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 --2- 172.21.15.105:0/3925478965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc107ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009990 tx=0x7f51f0009ca0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 -- 172.21.15.105:0/3925478965 shutdown_connections 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 --2- 172.21.15.105:0/3925478965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51fc107ff0 0x7f51fc10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 --2- 172.21.15.105:0/3925478965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 --2- 172.21.15.105:0/3925478965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc107ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 -- 172.21.15.105:0/3925478965 >> 172.21.15.105:0/3925478965 conn(0x7f51fc0fc7c0 msgr2=0x7f51fc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 -- 172.21.15.105:0/3925478965 shutdown_connections 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.676+0000 7f5201cb7700 1 -- 172.21.15.105:0/3925478965 wait complete. 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 Processor -- start 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 -- start start 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc1026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51fc107ff0 0x7f51fc101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10dfd0 con 0x7f51fc107ff0 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10de50 con 0x7f51fc1056c0 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f5201cb7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10e150 con 0x7f51fc102d90 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f51faffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f51faffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48648/0 (socket says 172.21.15.105:48648) 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f51faffd700 1 -- 172.21.15.105:0/2305985099 learned_addr learned my addr 172.21.15.105:0/2305985099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-16T10:02:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.677+0000 7f51fb7fe700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc1026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 -- 172.21.15.105:0/2305985099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 msgr2=0x7f51fc1026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc1026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 -- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51fc107ff0 msgr2=0x7f51fc101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51fc107ff0 0x7f51fc101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 -- 172.21.15.105:0/2305985099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc101b50 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51fb7fe700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51faffd700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005910 tx=0x7f51f00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0027070 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f0022470 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc1a2790 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000f6c0 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.678+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a2cc0 con 0x7f51fc1056c0 2024-03-16T10:02:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.679+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51fc068f90 con 0x7f51fc1056c0 2024-03-16T10:02:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.680+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f51f002a070 con 0x7f51fc1056c0 2024-03-16T10:02:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.681+0000 7f51f8ff9700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 0x7f51e40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:02:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.681+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f51f00cf250 con 0x7f51fc1056c0 2024-03-16T10:02:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.681+0000 7f51fb7fe700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 0x7f51e40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:02:54.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.682+0000 7f51fb7fe700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 0x7f51e40a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0099d0 tx=0x7f51ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:02:54.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.685+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f51f0031070 con 0x7f51fc1056c0 2024-03-16T10:02:54.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:02:54.888+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f51fc0627d0 con 0x7f51e40a2890 2024-03-16T10:02:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:55 smithi194 ceph-mon[36445]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:55 smithi194 ceph-mon[36445]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:55.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:55 smithi039 ceph-mon[31510]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:55.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:55 smithi039 ceph-mon[31510]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:55.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:55.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:55.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:55.519 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:55 smithi105 ceph-mon[36559]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:55.519 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:55 smithi105 ceph-mon[36559]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:02:55.519 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:02:55.519 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:02:55.519 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi039 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 5697M 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.591 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.592 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:56.592 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.592 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.592 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:56 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi039 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 5697M 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:56 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi039 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 5697M 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:56.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:02:56.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:02:56.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:02:56.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:56 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:02:58.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:58.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/582146703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]': finished 2024-03-16T10:02:58.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T10:02:58.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:02:58.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/582146703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]': finished 2024-03-16T10:02:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T10:02:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/582146703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]: dispatch 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b5763b-c967-49ad-a8d6-e37f49999f9c"}]': finished 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T10:02:58.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:02:59.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/62857079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:59.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:58 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/62857079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:02:59.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:58 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/62857079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:02:59 smithi039 ceph-mon[31510]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:00.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:02:59 smithi105 ceph-mon[36559]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:00.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:02:59 smithi194 ceph-mon[36445]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:01 smithi039 ceph-mon[31510]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:01 smithi105 ceph-mon[36559]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:01 smithi194 ceph-mon[36445]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:04.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:03 smithi039 ceph-mon[31510]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:04.056 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:03 smithi105 ceph-mon[36559]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:04.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:03 smithi194 ceph-mon[36445]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:05.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T10:03:05.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:05.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T10:03:05.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T10:03:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:05.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:05 smithi105 ceph-mon[36559]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:05.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:05 smithi105 ceph-mon[36559]: Deploying daemon osd.7 on smithi105 2024-03-16T10:03:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:05 smithi194 ceph-mon[36445]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:05 smithi194 ceph-mon[36445]: Deploying daemon osd.7 on smithi105 2024-03-16T10:03:06.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:05 smithi039 ceph-mon[31510]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:06.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:05 smithi039 ceph-mon[31510]: Deploying daemon osd.7 on smithi105 2024-03-16T10:03:08.017 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:07 smithi105 ceph-mon[36559]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:07 smithi194 ceph-mon[36445]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:07 smithi039 ceph-mon[31510]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:10.019 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:09 smithi105 ceph-mon[36559]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:09 smithi194 ceph-mon[36445]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:10.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:09 smithi039 ceph-mon[31510]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:11 smithi105 ceph-mon[36559]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:11 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:11 smithi194 ceph-mon[36445]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:11 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:11 smithi039 ceph-mon[31510]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:11 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:13.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:13 smithi039 ceph-mon[31510]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:13 smithi105 ceph-mon[36559]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:13 smithi194 ceph-mon[36445]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.725+0000 7f51f8ff9700 1 -- 172.21.15.105:0/2305985099 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51fc0627d0 con 0x7f51e40a2890 2024-03-16T10:03:13.726 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 7 on host 'smithi105' 2024-03-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 msgr2=0x7f51e40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 0x7f51e40a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0099d0 tx=0x7f51ec008040 comp rx=0 tx=0).stop 2024-03-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 msgr2=0x7f51fc100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005910 tx=0x7f51f00047b0 comp rx=0 tx=0).stop 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 shutdown_connections 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f51e40a2890 0x7f51e40a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51fc107ff0 0x7f51fc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51fc102d90 0x7f51fc1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 --2- 172.21.15.105:0/2305985099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51fc1056c0 0x7f51fc100d40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 >> 172.21.15.105:0/2305985099 conn(0x7f51fc0fc7c0 msgr2=0x7f51fc1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 shutdown_connections 2024-03-16T10:03:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:03:13.727+0000 7f5201cb7700 1 -- 172.21.15.105:0/2305985099 wait complete. 2024-03-16T10:03:14.414 DEBUG:teuthology.orchestra.run.smithi105:osd.7> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.7.service 2024-03-16T10:03:14.417 INFO:tasks.cephadm:Deploying osd.8 on smithi194 with /dev/vg_nvme/lv_2... 2024-03-16T10:03:14.417 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T10:03:14.664 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:15.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:15.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:15.783 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:15 smithi194 ceph-mon[36445]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-mon[36559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:15.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:03:15.707+0000 7f2a8fc69700 -1 osd.7 0 waiting for initial osdmap 2024-03-16T10:03:15.866 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:03:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:03:15.715+0000 7f2a8ba5c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:03:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:16.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T10:03:16.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: from='osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:16.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T10:03:16.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:16.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:15 smithi039 ceph-mon[31510]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114653 s, 915 MB/s 2024-03-16T10:03:16.643 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] boot 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T10:03:16.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:16 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] boot 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T10:03:16.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:16 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: osdmap e51: 8 total, 7 up, 8 in 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: osd.7 [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] boot 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T10:03:17.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:16 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:03:17.275 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-03-16T10:03:17.517 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:03:17.885 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:17 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:03:17.885 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:17 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:03:17.885 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:17 smithi194 ceph-mon[36445]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:17.885 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:17 smithi194 ceph-mon[36445]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T10:03:17.980 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:17 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:03:17.980 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:17 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:03:17.980 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:17 smithi105 ceph-mon[36559]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:17.981 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:17 smithi105 ceph-mon[36559]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T10:03:18.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:17 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:03:18.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:17 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:03:18.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:17 smithi039 ceph-mon[31510]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T10:03:18.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:17 smithi039 ceph-mon[31510]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.136+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/910066812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08103ea0 msgr2=0x7f0c0810a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.136+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/910066812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08103ea0 0x7f0c0810a750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000b0a0 tx=0x7f0c0000b3b0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/910066812 shutdown_connections 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/910066812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08103ea0 0x7f0c0810a750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/910066812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08102300 0x7f0c08102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/910066812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c08103500 0x7f0c08103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/910066812 >> 172.21.15.194:0/910066812 conn(0x7f0c080fd8e0 msgr2=0x7f0c080ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:03:19.140 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/910066812 shutdown_connections 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/910066812 wait complete. 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 Processor -- start 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.137+0000 7f0c0c8fa700 1 -- start start 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08103500 0x7f0c08198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c08103ea0 0x7f0c0819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810dfe0 con 0x7f0c08103ea0 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810de60 con 0x7f0c08103500 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0810e160 con 0x7f0c08102300 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:40720/0 (socket says 172.21.15.194:40720) 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 -- 172.21.15.194:0/2457179802 learned_addr learned my addr 172.21.15.194:0/2457179802 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c05d9b700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08103500 0x7f0c08198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 -- 172.21.15.194:0/2457179802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08103500 msgr2=0x7f0c08198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:19.141 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08103500 0x7f0c08198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 -- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c08103ea0 msgr2=0x7f0c0819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c08103ea0 0x7f0c0819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 -- 172.21.15.194:0/2457179802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c081a0290 con 0x7f0c08102300 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0659c700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b530 tx=0x7f0bf800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8015020 con 0x7f0c08102300 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c081a0580 con 0x7f0c08102300 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8011d10 con 0x7f0c08102300 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800f2e0 con 0x7f0c08102300 2024-03-16T10:03:19.142 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.138+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a0b40 con 0x7f0c08102300 2024-03-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.139+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c08068f90 con 0x7f0c08102300 2024-03-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.140+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0bf8018020 con 0x7f0c08102300 2024-03-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.141+0000 7f0bf77fe700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 0x7f0bf00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.141+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f0bf80cdab0 con 0x7f0c08102300 2024-03-16T10:03:19.143 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.141+0000 7f0c05d9b700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 0x7f0bf00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:19.144 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.142+0000 7f0c05d9b700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 0x7f0bf00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009a10 tx=0x7f0bfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:03:19.145 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.144+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0bf801f020 con 0x7f0c08102300 2024-03-16T10:03:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi105 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: pgmap v196: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:19 smithi194 ceph-mon[36445]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T10:03:19.351 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:19.349+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0c08062710 con 0x7f0bf00a2890 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi105 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-16T10:03:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: pgmap v196: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:19 smithi105 ceph-mon[36559]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T10:03:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi105 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: pgmap v196: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:19.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:19.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:19.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:19.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:19.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:19 smithi039 ceph-mon[31510]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T10:03:20.236 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-16T10:03:20.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:20.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:20.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:20.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:20.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:20 smithi194 ceph-mon[36445]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:20 smithi105 ceph-mon[36559]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T10:03:20.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-16T10:03:20.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:20.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:20.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:20.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:20.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:20 smithi039 ceph-mon[31510]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T10:03:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:21 smithi105 ceph-mon[36559]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:21 smithi105 ceph-mon[36559]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T10:03:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:21 smithi194 ceph-mon[36445]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:21 smithi194 ceph-mon[36445]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T10:03:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:21 smithi039 ceph-mon[31510]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:21 smithi039 ceph-mon[31510]: osdmap e56: 8 total, 8 up, 8 in 2024-03-16T10:03:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:22 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/3986791588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:22 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:22 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]': finished 2024-03-16T10:03:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:22 smithi194 ceph-mon[36445]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T10:03:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:22 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:22.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:22 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/3986791588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:22 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:22 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]': finished 2024-03-16T10:03:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:22 smithi105 ceph-mon[36559]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T10:03:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:22 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/3986791588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:22 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]: dispatch 2024-03-16T10:03:23.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:22 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e0fa1af-48e7-4764-ad45-390850e2bc7a"}]': finished 2024-03-16T10:03:23.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:22 smithi039 ceph-mon[31510]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T10:03:23.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:22 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:23 smithi105 ceph-mon[36559]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:23 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/2832244000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:23 smithi105 ceph-mon[36559]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-16T10:03:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:23 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:03:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:23 smithi194 ceph-mon[36445]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:23 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/2832244000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:23 smithi194 ceph-mon[36445]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-16T10:03:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:23 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:03:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:23 smithi039 ceph-mon[31510]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:23 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/2832244000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:23 smithi039 ceph-mon[31510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-16T10:03:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:23 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:03:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:25 smithi105 ceph-mon[36559]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:25 smithi194 ceph-mon[36445]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:26.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:25 smithi039 ceph-mon[31510]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:27 smithi105 ceph-mon[36559]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:27 smithi194 ceph-mon[36445]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:27 smithi039 ceph-mon[31510]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:29 smithi194 ceph-mon[36445]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T10:03:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:30.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:29 smithi039 ceph-mon[31510]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:30.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T10:03:30.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:29 smithi105 ceph-mon[36559]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T10:03:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:30 smithi194 ceph-mon[36445]: Deploying daemon osd.8 on smithi194 2024-03-16T10:03:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:30 smithi039 ceph-mon[31510]: Deploying daemon osd.8 on smithi194 2024-03-16T10:03:31.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:30 smithi105 ceph-mon[36559]: Deploying daemon osd.8 on smithi194 2024-03-16T10:03:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:31 smithi194 ceph-mon[36445]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:32.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:31 smithi039 ceph-mon[31510]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:32.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:31 smithi105 ceph-mon[36559]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:33.916 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:33 smithi105 ceph-mon[36559]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:33.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:33 smithi194 ceph-mon[36445]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:33 smithi039 ceph-mon[31510]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:36.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:36 smithi105 ceph-mon[36559]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:36.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:36 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:36 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:36 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:36.426 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:36 smithi194 ceph-mon[36445]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:36.426 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:36 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.426 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:36 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.426 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:36 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:36 smithi039 ceph-mon[31510]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:36 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:36 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:36 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:38.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:38 smithi194 ceph-mon[36445]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:38.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:38 smithi105 ceph-mon[36559]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:38.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:38 smithi039 ceph-mon[31510]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:38.989 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 8 on host 'smithi194' 2024-03-16T10:03:38.990 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.988+0000 7f0bf77fe700 1 -- 172.21.15.194:0/2457179802 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c08062710 con 0x7f0bf00a2890 2024-03-16T10:03:38.991 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 msgr2=0x7f0bf00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:38.991 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 0x7f0bf00a4d50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009a10 tx=0x7f0bfc009190 comp rx=0 tx=0).stop 2024-03-16T10:03:38.991 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 msgr2=0x7f0c08198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:38.991 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b530 tx=0x7f0bf800b840 comp rx=0 tx=0).stop 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 shutdown_connections 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bf00a2890 0x7f0bf00a4d50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c08103ea0 0x7f0c0819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c08102300 0x7f0c08198630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 --2- 172.21.15.194:0/2457179802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c08103500 0x7f0c08198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 >> 172.21.15.194:0/2457179802 conn(0x7f0c080fd8e0 msgr2=0x7f0c08106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 shutdown_connections 2024-03-16T10:03:38.992 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:03:38.991+0000 7f0c0c8fa700 1 -- 172.21.15.194:0/2457179802 wait complete. 2024-03-16T10:03:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:39 smithi039 ceph-mon[31510]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:39 smithi105 ceph-mon[36559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.682 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:39.682 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.683 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:39 smithi194 ceph-mon[36445]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T10:03:39.708 DEBUG:teuthology.orchestra.run.smithi194:osd.8> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.8.service 2024-03-16T10:03:39.711 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2024-03-16T10:03:39.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T10:03:40.181 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:03:41.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:40 smithi194 ceph-mon[36445]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T10:03:41.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:40 smithi194 ceph-mon[36445]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:40 smithi194 ceph-mon[36445]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T10:03:41.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:41.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:40 smithi194 ceph-mon[36445]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:40 smithi039 ceph-mon[31510]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T10:03:41.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:40 smithi039 ceph-mon[31510]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:40 smithi039 ceph-mon[31510]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T10:03:41.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:41.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:40 smithi039 ceph-mon[31510]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:40 smithi105 ceph-mon[36559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T10:03:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:40 smithi105 ceph-mon[36559]: from='osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:40 smithi105 ceph-mon[36559]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T10:03:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:40 smithi105 ceph-mon[36559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:03:41.706 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:03:41.418+0000 7fc29c8e6700 -1 osd.8 0 waiting for initial osdmap 2024-03-16T10:03:41.706 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:03:41.432+0000 7fc2946d1700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:03:42.151 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:03:42.151 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: from='osd.8 ' entity='osd.8' 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] boot 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T10:03:42.152 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: from='osd.8 ' entity='osd.8' 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] boot 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T10:03:42.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: osdmap e59: 9 total, 8 up, 9 in 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: from='osd.8 ' entity='osd.8' 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: osd.8 [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] boot 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T10:03:42.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0361882 s, 290 MB/s 2024-03-16T10:03:43.140 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-03-16T10:03:43.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi194 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:43.834 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi194 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T10:03:43.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:43.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:43.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi194 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:03:43.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:03:44.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-03-16T10:03:44.498 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:03:44.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:44 smithi039 ceph-mon[31510]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T10:03:44.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:44 smithi105 ceph-mon[36559]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T10:03:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:44 smithi194 ceph-mon[36445]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T10:03:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:45 smithi105 ceph-mon[36559]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:45.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:45 smithi105 ceph-mon[36559]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T10:03:45.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:45 smithi194 ceph-mon[36445]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:45.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:45 smithi194 ceph-mon[36445]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T10:03:45.976 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:45 smithi039 ceph-mon[31510]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:45.977 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:45 smithi039 ceph-mon[31510]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T10:03:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 -- 172.21.15.39:0/998950911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 msgr2=0x7f18e410a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 --2- 172.21.15.39:0/998950911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e410a460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b0a0 tx=0x7f18dc00b3b0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 -- 172.21.15.39:0/998950911 shutdown_connections 2024-03-16T10:03:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 --2- 172.21.15.39:0/998950911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e410a460 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 --2- 172.21.15.39:0/998950911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 0x7f18e41051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 --2- 172.21.15.39:0/998950911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 -- 172.21.15.39:0/998950911 >> 172.21.15.39:0/998950911 conn(0x7f18e40fc7a0 msgr2=0x7f18e40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:03:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 -- 172.21.15.39:0/998950911 shutdown_connections 2024-03-16T10:03:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.828+0000 7f18e8e00700 1 -- 172.21.15.39:0/998950911 wait complete. 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.829+0000 7f18e8e00700 1 Processor -- start 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 -- start start 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 0x7f18e419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410df60 con 0x7f18e4108050 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410dde0 con 0x7f18e4102df0 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e8e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f18e410e0e0 con 0x7f18e4105720 2024-03-16T10:03:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e2800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18dbe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18dbe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:49850/0 (socket says 172.21.15.39:49850) 2024-03-16T10:03:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18dbe00700 1 -- 172.21.15.39:0/970590278 learned_addr learned my addr 172.21.15.39:0/970590278 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:03:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.830+0000 7f18e1e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 0x7f18e419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 -- 172.21.15.39:0/970590278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 msgr2=0x7f18e419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 0x7f18e419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 -- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 msgr2=0x7f18e41a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e41a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 -- 172.21.15.39:0/970590278 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e41a2360 con 0x7f18e4105720 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18e2800700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.831+0000 7f18dbe00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00eb60 tx=0x7f18cc00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.832+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00cbd0 con 0x7f18e4105720 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.832+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18cc004500 con 0x7f18e4105720 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.832+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e41a2650 con 0x7f18e4105720 2024-03-16T10:03:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.832+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc010430 con 0x7f18e4105720 2024-03-16T10:03:46.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.832+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e41a2be0 con 0x7f18e4105720 2024-03-16T10:03:46.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.833+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e4068f90 con 0x7f18e4105720 2024-03-16T10:03:46.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.835+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f18cc003680 con 0x7f18e4105720 2024-03-16T10:03:46.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.836+0000 7f18daa00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 0x7f18c80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:03:46.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.836+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f18cc0ce2c0 con 0x7f18e4105720 2024-03-16T10:03:46.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.837+0000 7f18e1e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 0x7f18c80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:03:46.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.837+0000 7f18e1e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 0x7f18c80a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c60 tx=0x7f18d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:03:46.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:46.841+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f18cc00cd70 con 0x7f18e4105720 2024-03-16T10:03:46.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:46 smithi105 ceph-mon[36559]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T10:03:46.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:46 smithi194 ceph-mon[36445]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T10:03:47.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:46 smithi039 ceph-mon[31510]: osdmap e64: 9 total, 9 up, 9 in 2024-03-16T10:03:47.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:03:47.094+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f18e4062710 con 0x7f18c80a2910 2024-03-16T10:03:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:47 smithi194 ceph-mon[36445]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:47 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:47 smithi039 ceph-mon[31510]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:47 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:48.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:47 smithi105 ceph-mon[36559]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:03:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:03:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:47 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:03:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:48 smithi039 ceph-mon[31510]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:48.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:48 smithi194 ceph-mon[36445]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:49.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:48 smithi105 ceph-mon[36559]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:03:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:49 smithi039 ceph-mon[31510]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:03:49.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:49 smithi194 ceph-mon[36445]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:03:50.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:49 smithi105 ceph-mon[36559]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:03:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:51 smithi194 ceph-mon[36445]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:03:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]: dispatch 2024-03-16T10:03:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]': finished 2024-03-16T10:03:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:51 smithi194 ceph-mon[36445]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T10:03:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:51 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:03:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:51 smithi039 ceph-mon[31510]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:03:52.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]: dispatch 2024-03-16T10:03:52.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]': finished 2024-03-16T10:03:52.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:51 smithi039 ceph-mon[31510]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T10:03:52.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:51 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:03:52.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:51 smithi105 ceph-mon[36559]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-16T10:03:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]: dispatch 2024-03-16T10:03:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4171687638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c112bf93-06c2-409e-8f31-837fd3d2b600"}]': finished 2024-03-16T10:03:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:51 smithi105 ceph-mon[36559]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T10:03:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:51 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:03:52.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2026423892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:53.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2026423892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:53.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2026423892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:03:53.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:53 smithi039 ceph-mon[31510]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:03:53.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:53 smithi194 ceph-mon[36445]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:03:54.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:53 smithi105 ceph-mon[36559]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:03:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:55 smithi194 ceph-mon[36445]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:03:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:55 smithi039 ceph-mon[31510]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:03:56.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:55 smithi105 ceph-mon[36559]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:03:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:57 smithi039 ceph-mon[31510]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:03:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:57 smithi194 ceph-mon[36445]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:03:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:57 smithi105 ceph-mon[36559]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:03:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:03:59 smithi039 ceph-mon[31510]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:03:59.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:03:59 smithi194 ceph-mon[36445]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:03:59 smithi105 ceph-mon[36559]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:00.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T10:04:00.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T10:04:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:01.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T10:04:01.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:01 smithi194 ceph-mon[36445]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:01 smithi194 ceph-mon[36445]: Deploying daemon osd.9 on smithi039 2024-03-16T10:04:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:01 smithi039 ceph-mon[31510]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:01 smithi039 ceph-mon[31510]: Deploying daemon osd.9 on smithi039 2024-03-16T10:04:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:01 smithi105 ceph-mon[36559]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:01 smithi105 ceph-mon[36559]: Deploying daemon osd.9 on smithi039 2024-03-16T10:04:03.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:03 smithi039 ceph-mon[31510]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:03 smithi194 ceph-mon[36445]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:04.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:03 smithi105 ceph-mon[36559]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:05.985 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:05 smithi039 ceph-mon[31510]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:06.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:05 smithi105 ceph-mon[36559]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:06.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:05 smithi194 ceph-mon[36445]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:08.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:07 smithi039 ceph-mon[31510]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:08.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:07 smithi105 ceph-mon[36559]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:07 smithi194 ceph-mon[36445]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:09.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:09 smithi105 ceph-mon[36559]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:09.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:09.872 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:09 smithi039 ceph-mon[31510]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:09.872 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.872 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.873 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:09.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:09 smithi194 ceph-mon[36445]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:09.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:09.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:11.663 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:11 smithi039 ceph-mon[31510]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:11 smithi105 ceph-mon[36559]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:11 smithi194 ceph-mon[36445]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:13.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:13 smithi039 ceph-mon[31510]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:13.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:13 smithi039 ceph-mon[31510]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:13.468 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:13 smithi039 ceph-mon[31510]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:13.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.547+0000 7f18daa00700 1 -- 172.21.15.39:0/970590278 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f18e4062710 con 0x7f18c80a2910 2024-03-16T10:04:13.549 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2024-03-16T10:04:13.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.550+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 msgr2=0x7f18c80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:13.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.550+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 0x7f18c80a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009c60 tx=0x7f18d4009260 comp rx=0 tx=0).stop 2024-03-16T10:04:13.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.550+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 msgr2=0x7f18e419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:13.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.550+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00eb60 tx=0x7f18cc00ee70 comp rx=0 tx=0).stop 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 shutdown_connections 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f18c80a2910 0x7f18c80a4dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e4108050 0x7f18e41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18e4102df0 0x7f18e419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 --2- 172.21.15.39:0/970590278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18e4105720 0x7f18e419ac40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 >> 172.21.15.39:0/970590278 conn(0x7f18e40fc7a0 msgr2=0x7f18e4102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 shutdown_connections 2024-03-16T10:04:13.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:04:13.551+0000 7f18e8e00700 1 -- 172.21.15.39:0/970590278 wait complete. 2024-03-16T10:04:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:13 smithi105 ceph-mon[36559]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:13 smithi105 ceph-mon[36559]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:13 smithi105 ceph-mon[36559]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:13 smithi194 ceph-mon[36445]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:13.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:13 smithi194 ceph-mon[36445]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:13.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:13 smithi194 ceph-mon[36445]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T10:04:14.270 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.9.service 2024-03-16T10:04:14.274 INFO:tasks.cephadm:Deploying osd.10 on smithi105 with /dev/vg_nvme/lv_1... 2024-03-16T10:04:14.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T10:04:14.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:14.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:14.520 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.609 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:14.610 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:14.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:15 smithi105 ceph-mon[36559]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:15 smithi105 ceph-mon[36559]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:04:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:15 smithi105 ceph-mon[36559]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T10:04:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:15 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:15.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:15 smithi039 ceph-mon[31510]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:15.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:15 smithi039 ceph-mon[31510]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:04:15.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:15 smithi039 ceph-mon[31510]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T10:04:15.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:15.676 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:15 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:15 smithi194 ceph-mon[36445]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:15 smithi194 ceph-mon[36445]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-03-16T10:04:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:15 smithi194 ceph-mon[36445]: osdmap e67: 10 total, 9 up, 10 in 2024-03-16T10:04:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:15 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:16.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:16 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:04:16.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:16 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:04:16.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:16 smithi105 ceph-mon[36559]: osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] boot 2024-03-16T10:04:16.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:16 smithi105 ceph-mon[36559]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T10:04:16.524 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:16 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114826 s, 913 MB/s 2024-03-16T10:04:16.620 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-16T10:04:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:16 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:04:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:16 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:04:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:16 smithi194 ceph-mon[36445]: osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] boot 2024-03-16T10:04:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:16 smithi194 ceph-mon[36445]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T10:04:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:16 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:16.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:16 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:04:16.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:16 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:04:16.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:16 smithi039 ceph-mon[31510]: osd.9 [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] boot 2024-03-16T10:04:16.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:16 smithi039 ceph-mon[31510]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T10:04:16.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:16 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:04:17.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-03-16T10:04:17.564 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:04:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:17 smithi105 ceph-mon[36559]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:17.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:17 smithi105 ceph-mon[36559]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T10:04:17.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:17 smithi194 ceph-mon[36445]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:17.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:17 smithi194 ceph-mon[36445]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T10:04:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:17 smithi039 ceph-mon[31510]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T10:04:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:17 smithi039 ceph-mon[31510]: osdmap e69: 10 total, 10 up, 10 in 2024-03-16T10:04:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 -- 172.21.15.105:0/3846127316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70105720 msgr2=0x7f1a70107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 --2- 172.21.15.105:0/3846127316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70105720 0x7f1a70107b10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60009ca0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 -- 172.21.15.105:0/3846127316 shutdown_connections 2024-03-16T10:04:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 --2- 172.21.15.105:0/3846127316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70108050 0x7f1a7010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 --2- 172.21.15.105:0/3846127316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70102df0 0x7f1a701051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 --2- 172.21.15.105:0/3846127316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70105720 0x7f1a70107b10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 -- 172.21.15.105:0/3846127316 >> 172.21.15.105:0/3846127316 conn(0x7f1a700fc7a0 msgr2=0x7f1a700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 -- 172.21.15.105:0/3846127316 shutdown_connections 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.252+0000 7f1a77416700 1 -- 172.21.15.105:0/3846127316 wait complete. 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 Processor -- start 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 -- start start 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70102df0 0x7f1a70102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 0x7f1a70100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010df50 con 0x7f1a70102df0 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010ddd0 con 0x7f1a70108050 2024-03-16T10:04:19.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a77416700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010e0d0 con 0x7f1a70105720 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43138/0 (socket says 172.21.15.105:43138) 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 -- 172.21.15.105:0/3837289327 learned_addr learned my addr 172.21.15.105:0/3837289327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a749b1700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 0x7f1a70100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a751b2700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70102df0 0x7f1a70102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 -- 172.21.15.105:0/3837289327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 msgr2=0x7f1a70100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 0x7f1a70100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 -- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70102df0 msgr2=0x7f1a70102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70102df0 0x7f1a70102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a759b3700 1 -- 172.21.15.105:0/3837289327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a70101b90 con 0x7f1a70108050 2024-03-16T10:04:19.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.253+0000 7f1a749b1700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 0x7f1a70100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a759b3700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00b9e0 tx=0x7f1a6c00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c019070 con 0x7f1a70108050 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701a2730 con 0x7f1a70108050 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a6c014440 con 0x7f1a70108050 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0125a0 con 0x7f1a70108050 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.254+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701a2cc0 con 0x7f1a70108050 2024-03-16T10:04:19.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.255+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a70068f90 con 0x7f1a70108050 2024-03-16T10:04:19.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.256+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1a6c01c070 con 0x7f1a70108050 2024-03-16T10:04:19.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.256+0000 7f1a667fc700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 0x7f1a5c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:19.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.257+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f1a6c0cdb30 con 0x7f1a70108050 2024-03-16T10:04:19.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.257+0000 7f1a751b2700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 0x7f1a5c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:19.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.257+0000 7f1a751b2700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 0x7f1a5c0a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a68005fd0 tx=0x7f1a6800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:04:19.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.260+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1a6c09b3b0 con 0x7f1a70108050 2024-03-16T10:04:19.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:19.468+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1a70062710 con 0x7f1a5c0a2890 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:19.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:19.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:19.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi039 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 4272M 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:20.359 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.360 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:20.360 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi039 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 4272M 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi039 2024-03-16T10:04:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 4272M 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:21 smithi105 ceph-mon[36559]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:21 smithi194 ceph-mon[36445]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:21 smithi039 ceph-mon[31510]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:22 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/147196381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:22 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:22 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]': finished 2024-03-16T10:04:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:22 smithi105 ceph-mon[36559]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T10:04:22.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:22 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:22 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/147196381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:22 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:22 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]': finished 2024-03-16T10:04:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:22 smithi194 ceph-mon[36445]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T10:04:22.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:22 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:23.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:22 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/147196381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:23.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:22 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]: dispatch 2024-03-16T10:04:23.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:22 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44ef9e2-7391-4298-a185-199839c37867"}]': finished 2024-03-16T10:04:23.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:22 smithi039 ceph-mon[31510]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T10:04:23.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:22 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:23 smithi105 ceph-mon[36559]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:23 smithi105 ceph-mon[36559]: from='client.? 172.21.15.105:0/3709418946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:23 smithi194 ceph-mon[36445]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:23.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:23 smithi194 ceph-mon[36445]: from='client.? 172.21.15.105:0/3709418946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:23 smithi039 ceph-mon[31510]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:23 smithi039 ceph-mon[31510]: from='client.? 172.21.15.105:0/3709418946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:25.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:25 smithi105 ceph-mon[36559]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:25 smithi194 ceph-mon[36445]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:26.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:25 smithi039 ceph-mon[31510]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:27.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:27 smithi105 ceph-mon[36559]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:27 smithi194 ceph-mon[36445]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:27 smithi039 ceph-mon[31510]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:29 smithi194 ceph-mon[36445]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:30.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:29 smithi039 ceph-mon[31510]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:30.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:29 smithi105 ceph-mon[36559]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:30.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:30 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T10:04:30.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:30 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:30.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:30 smithi105 ceph-mon[36559]: Deploying daemon osd.10 on smithi105 2024-03-16T10:04:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:30 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T10:04:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:30 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:30.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:30 smithi194 ceph-mon[36445]: Deploying daemon osd.10 on smithi105 2024-03-16T10:04:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T10:04:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:31.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:30 smithi039 ceph-mon[31510]: Deploying daemon osd.10 on smithi105 2024-03-16T10:04:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:31 smithi194 ceph-mon[36445]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:32.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:31 smithi039 ceph-mon[31510]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:32.009 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:31 smithi105 ceph-mon[36559]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:33.721 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:33 smithi105 ceph-mon[36559]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:33.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:33 smithi194 ceph-mon[36445]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:34.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:33 smithi039 ceph-mon[31510]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:35 smithi105 ceph-mon[36559]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:35 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:35 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:35 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:35 smithi194 ceph-mon[36445]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:35 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:35 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:35 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:35 smithi039 ceph-mon[31510]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:35 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:35 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:35 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:37 smithi105 ceph-mon[36559]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:37 smithi194 ceph-mon[36445]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:37 smithi039 ceph-mon[31510]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:39.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.459+0000 7f1a667fc700 1 -- 172.21.15.105:0/3837289327 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1a70062710 con 0x7f1a5c0a2890 2024-03-16T10:04:39.461 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 10 on host 'smithi105' 2024-03-16T10:04:39.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 msgr2=0x7f1a5c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 0x7f1a5c0a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a68005fd0 tx=0x7f1a6800a560 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 msgr2=0x7f1a701012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00b9e0 tx=0x7f1a6c00bda0 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 shutdown_connections 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1a5c0a2890 0x7f1a5c0a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a70108050 0x7f1a701012e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a70102df0 0x7f1a70102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 --2- 172.21.15.105:0/3837289327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a70105720 0x7f1a70100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 >> 172.21.15.105:0/3837289327 conn(0x7f1a700fc7a0 msgr2=0x7f1a70103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.461+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 shutdown_connections 2024-03-16T10:04:39.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-16T10:04:39.462+0000 7f1a77416700 1 -- 172.21.15.105:0/3837289327 wait complete. 2024-03-16T10:04:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:39.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:04:40.152 DEBUG:teuthology.orchestra.run.smithi105:osd.10> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.10.service 2024-03-16T10:04:40.154 INFO:tasks.cephadm:Deploying osd.11 on smithi194 with /dev/vg_nvme/lv_1... 2024-03-16T10:04:40.154 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T10:04:40.399 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:04:40.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:40 smithi194 ceph-mon[36445]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T10:04:40.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:40 smithi194 ceph-mon[36445]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T10:04:40.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:40 smithi194 ceph-mon[36445]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:40.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:40.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:40 smithi194 ceph-mon[36445]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:40 smithi039 ceph-mon[31510]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T10:04:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:40 smithi039 ceph-mon[31510]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T10:04:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:40 smithi039 ceph-mon[31510]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:40 smithi039 ceph-mon[31510]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:41.019 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-mon[36559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T10:04:41.019 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-mon[36559]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T10:04:41.020 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-mon[36559]: from='osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:41.020 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:41.020 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-mon[36559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-16T10:04:41.020 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:04:40.694+0000 7f3485dc5700 -1 osd.10 0 waiting for initial osdmap 2024-03-16T10:04:41.020 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:04:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:04:40.707+0000 7f3481bb8700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] boot 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T10:04:41.801 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:41 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] boot 2024-03-16T10:04:42.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T10:04:42.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:41 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: osdmap e72: 11 total, 10 up, 11 in 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: osd.10 [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] boot 2024-03-16T10:04:42.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T10:04:42.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:41 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:04:42.487 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T10:04:42.487 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T10:04:42.488 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T10:04:42.488 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T10:04:42.488 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114697 s, 914 MB/s 2024-03-16T10:04:42.488 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T10:04:42.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:42 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:04:42.811 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:42 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:04:42.812 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:42 smithi105 ceph-mon[36559]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T10:04:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:42 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:04:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:42 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:04:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:42 smithi039 ceph-mon[31510]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T10:04:43.199 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:42 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:04:43.199 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:42 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:04:43.199 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:42 smithi194 ceph-mon[36445]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T10:04:43.226 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2024-03-16T10:04:43.467 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:04:43.860 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:43.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:43 smithi194 ceph-mon[36445]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T10:04:44.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:43 smithi039 ceph-mon[31510]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:04:44.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:43 smithi105 ceph-mon[36559]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T10:04:44.941 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:44 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi105 2024-03-16T10:04:44.941 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:44 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-16T10:04:44.941 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:44 smithi194 ceph-mon[36445]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T10:04:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:44 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi105 2024-03-16T10:04:45.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:44 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-16T10:04:45.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:44 smithi039 ceph-mon[31510]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T10:04:45.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:44 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi105 2024-03-16T10:04:45.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:44 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-16T10:04:45.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:44 smithi105 ceph-mon[36559]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T10:04:45.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 -- 172.21.15.194:0/2549915611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103ea0 msgr2=0x7f92ac10a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 --2- 172.21.15.194:0/2549915611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103ea0 0x7f92ac10a750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b0a0 tx=0x7f92a400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 -- 172.21.15.194:0/2549915611 shutdown_connections 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 --2- 172.21.15.194:0/2549915611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103ea0 0x7f92ac10a750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 --2- 172.21.15.194:0/2549915611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac102300 0x7f92ac102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 --2- 172.21.15.194:0/2549915611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ac103500 0x7f92ac103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 -- 172.21.15.194:0/2549915611 >> 172.21.15.194:0/2549915611 conn(0x7f92ac0fd8e0 msgr2=0x7f92ac0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 -- 172.21.15.194:0/2549915611 shutdown_connections 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.146+0000 7f92b2f26700 1 -- 172.21.15.194:0/2549915611 wait complete. 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 Processor -- start 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 -- start start 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ac102300 0x7f92ac1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:45.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac103ea0 0x7f92ac19fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10df60 con 0x7f92ac102300 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10dde0 con 0x7f92ac103ea0 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92b2f26700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac10e0e0 con 0x7f92ac103500 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92abfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92abfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:42196/0 (socket says 172.21.15.194:42196) 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.147+0000 7f92abfff700 1 -- 172.21.15.194:0/1139374850 learned_addr learned my addr 172.21.15.194:0/1139374850 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 -- 172.21.15.194:0/1139374850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac103ea0 msgr2=0x7f92ac19fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92b14c3700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac103ea0 0x7f92ac19fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac103ea0 0x7f92ac19fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 -- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ac102300 msgr2=0x7f92ac1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ac102300 0x7f92ac1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:04:45.151 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 -- 172.21.15.194:0/1139374850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac1a0150 con 0x7f92ac103500 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92abfff700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f929800b560 tx=0x7f929800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298015020 con 0x7f92ac103500 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9298011d10 con 0x7f92ac103500 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac1a0440 con 0x7f92ac103500 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929800f2e0 con 0x7f92ac103500 2024-03-16T10:04:45.152 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.148+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac1a0a00 con 0x7f92ac103500 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.149+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ac068f90 con 0x7f92ac103500 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.150+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9298018020 con 0x7f92ac103500 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.151+0000 7f92a9ffb700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 0x7f929c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.151+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f92980cdc60 con 0x7f92ac103500 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.151+0000 7f92b0cc2700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 0x7f929c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:04:45.153 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.152+0000 7f92b0cc2700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 0x7f929c0a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92ac103360 tx=0x7f92a0005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:04:45.156 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.155+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f929801f020 con 0x7f92ac103500 2024-03-16T10:04:45.361 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:04:45.360+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f92ac062710 con 0x7f929c0a2890 2024-03-16T10:04:45.994 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:45 smithi194 ceph-mon[36445]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:45.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:45.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:45.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:45.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:45 smithi194 ceph-mon[36445]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T10:04:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:45 smithi039 ceph-mon[31510]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:45 smithi039 ceph-mon[31510]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T10:04:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:45 smithi105 ceph-mon[36559]: pgmap v261: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:04:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:04:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:45 smithi105 ceph-mon[36559]: osdmap e77: 11 total, 11 up, 11 in 2024-03-16T10:04:47.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:46 smithi039 ceph-mon[31510]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:47.023 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:46 smithi194 ceph-mon[36445]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:47.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:46 smithi105 ceph-mon[36559]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:04:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:47 smithi039 ceph-mon[31510]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:47 smithi105 ceph-mon[36559]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:48.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:47 smithi194 ceph-mon[36445]: pgmap v264: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:04:48.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:48 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/3806500392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:48.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:48 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:48.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:48 smithi194 ceph-mon[36445]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]': finished 2024-03-16T10:04:48.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:48 smithi194 ceph-mon[36445]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T10:04:48.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:48 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:04:49.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:48 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/3806500392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:49.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:48 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:49.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:48 smithi105 ceph-mon[36559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]': finished 2024-03-16T10:04:49.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:48 smithi105 ceph-mon[36559]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T10:04:49.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:48 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:04:49.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:48 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/3806500392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:49.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:48 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]: dispatch 2024-03-16T10:04:49.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:48 smithi039 ceph-mon[31510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a171911-8bf9-4ad4-ab72-98d5c48d0807"}]': finished 2024-03-16T10:04:49.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:48 smithi039 ceph-mon[31510]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T10:04:49.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:48 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:04:50.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:49 smithi105 ceph-mon[36559]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:04:50.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:49 smithi105 ceph-mon[36559]: from='client.? 172.21.15.194:0/1712224110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:50.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:49 smithi194 ceph-mon[36445]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:04:50.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:49 smithi194 ceph-mon[36445]: from='client.? 172.21.15.194:0/1712224110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:49 smithi039 ceph-mon[31510]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-16T10:04:50.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:49 smithi039 ceph-mon[31510]: from='client.? 172.21.15.194:0/1712224110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:04:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:51 smithi105 ceph-mon[36559]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:04:52.168 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:51 smithi194 ceph-mon[36445]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:04:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:51 smithi039 ceph-mon[31510]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:04:54.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:53 smithi105 ceph-mon[36559]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:04:54.159 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:53 smithi194 ceph-mon[36445]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:04:54.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:53 smithi039 ceph-mon[31510]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T10:04:56.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:55 smithi105 ceph-mon[36559]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:04:56.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T10:04:56.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:56.146 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:55 smithi194 ceph-mon[36445]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:04:56.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T10:04:56.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:55 smithi039 ceph-mon[31510]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T10:04:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T10:04:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:04:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:56 smithi039 ceph-mon[31510]: Deploying daemon osd.11 on smithi194 2024-03-16T10:04:57.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:56 smithi105 ceph-mon[36559]: Deploying daemon osd.11 on smithi194 2024-03-16T10:04:57.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:56 smithi194 ceph-mon[36445]: Deploying daemon osd.11 on smithi194 2024-03-16T10:04:58.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:57 smithi194 ceph-mon[36445]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:04:58.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:57 smithi105 ceph-mon[36559]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:04:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:57 smithi039 ceph-mon[31510]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:05:00.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:04:59 smithi105 ceph-mon[36559]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:00.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:04:59 smithi194 ceph-mon[36445]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:04:59 smithi039 ceph-mon[31510]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:02.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:01 smithi194 ceph-mon[36445]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:02.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:02.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:02.114 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:01 smithi105 ceph-mon[36559]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:02.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:01 smithi039 ceph-mon[31510]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:02.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:02.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:02.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:04.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:03 smithi105 ceph-mon[36559]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:04.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:03 smithi194 ceph-mon[36445]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:04.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:03 smithi039 ceph-mon[31510]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:05.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.398+0000 7f92a9ffb700 1 -- 172.21.15.194:0/1139374850 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f92ac062710 con 0x7f929c0a2890 2024-03-16T10:05:05.399 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 11 on host 'smithi194' 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 msgr2=0x7f929c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 0x7f929c0a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f92ac103360 tx=0x7f92a0005ca0 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 msgr2=0x7f92ac198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f929800b560 tx=0x7f929800b920 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 shutdown_connections 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f929c0a2890 0x7f929c0a4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92ac103ea0 0x7f92ac19fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ac102300 0x7f92ac1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 --2- 172.21.15.194:0/1139374850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92ac103500 0x7f92ac198a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.400+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 >> 172.21.15.194:0/1139374850 conn(0x7f92ac0fd8e0 msgr2=0x7f92ac1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.401+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 shutdown_connections 2024-03-16T10:05:05.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:05.401+0000 7f92b2f26700 1 -- 172.21.15.194:0/1139374850 wait complete. 2024-03-16T10:05:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:05.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:05.981 DEBUG:teuthology.orchestra.run.smithi194:osd.11> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.11.service 2024-03-16T10:05:05.985 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-16T10:05:05.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:06.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:06.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:06.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:06.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:05:06.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:06 smithi039 ceph-mon[31510]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T10:05:06.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:06 smithi039 ceph-mon[31510]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T10:05:06.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:06 smithi039 ceph-mon[31510]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:06.910 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:06.910 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:06 smithi039 ceph-mon[31510]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:07.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:06 smithi105 ceph-mon[36559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T10:05:07.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:06 smithi105 ceph-mon[36559]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T10:05:07.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:06 smithi105 ceph-mon[36559]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:07.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:07.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:06 smithi105 ceph-mon[36559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:07.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-mon[36445]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T10:05:07.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-mon[36445]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T10:05:07.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-mon[36445]: from='osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:07.147 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:07.148 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-mon[36445]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T10:05:07.148 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:05:06.836+0000 7f9d3f77c700 -1 osd.11 0 waiting for initial osdmap 2024-03-16T10:05:07.148 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:05:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:05:06.848+0000 7f9d3b56f700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:05:08.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:07 smithi105 ceph-mon[36559]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:08.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:07 smithi105 ceph-mon[36559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:05:08.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:07 smithi105 ceph-mon[36559]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T10:05:08.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:07 smithi039 ceph-mon[31510]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:08.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:07 smithi039 ceph-mon[31510]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:05:08.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:07 smithi039 ceph-mon[31510]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T10:05:08.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.202 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:07 smithi194 ceph-mon[36445]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:07 smithi194 ceph-mon[36445]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T10:05:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:07 smithi194 ceph-mon[36445]: osdmap e80: 12 total, 11 up, 12 in 2024-03-16T10:05:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:08.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 -- 172.21.15.39:0/2372798251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec102350 msgr2=0x7f0fec102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/2372798251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec102350 0x7f0fec102770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0099c0 tx=0x7f0fdc009cd0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 -- 172.21.15.39:0/2372798251 shutdown_connections 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/2372798251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 0x7f0fec10a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/2372798251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec102350 0x7f0fec102770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/2372798251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec103550 0x7f0fec1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 -- 172.21.15.39:0/2372798251 >> 172.21.15.39:0/2372798251 conn(0x7f0fec0fd8f0 msgr2=0x7f0fec0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 -- 172.21.15.39:0/2372798251 shutdown_connections 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.755+0000 7f0ff0a00700 1 -- 172.21.15.39:0/2372798251 wait complete. 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.756+0000 7f0ff0a00700 1 Processor -- start 2024-03-16T10:05:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.756+0000 7f0ff0a00700 1 -- start start 2024-03-16T10:05:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec103550 0x7f0fec19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 0x7f0fec1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec10df80 con 0x7f0fec103550 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec10de00 con 0x7f0fec103ef0 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0ff0a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fec10e100 con 0x7f0fec102350 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0fe9e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0fe9e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:50408/0 (socket says 172.21.15.39:50408) 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0fe9e00700 1 -- 172.21.15.39:0/943723884 learned_addr learned my addr 172.21.15.39:0/943723884 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0fe9400700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec103550 0x7f0fec19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.757+0000 7f0fea800700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 0x7f0fec1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 -- 172.21.15.39:0/943723884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 msgr2=0x7f0fec1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 0x7f0fec1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 -- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec103550 msgr2=0x7f0fec19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec103550 0x7f0fec19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 -- 172.21.15.39:0/943723884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fec1a2410 con 0x7f0fec102350 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe9e00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00bee0 tx=0x7f0fdc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc009e60 con 0x7f0fec102350 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc003e30 con 0x7f0fec102350 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fec1a26a0 con 0x7f0fec102350 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0173f0 con 0x7f0fec102350 2024-03-16T10:05:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.758+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fec1a2af0 con 0x7f0fec102350 2024-03-16T10:05:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.760+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fec068fc0 con 0x7f0fec102350 2024-03-16T10:05:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.761+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0fdc00f660 con 0x7f0fec102350 2024-03-16T10:05:08.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.763+0000 7f0fe2a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 0x7f0fd00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:08.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.763+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0fdc0cf3e0 con 0x7f0fec102350 2024-03-16T10:05:08.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.763+0000 7f0fe9400700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 0x7f0fd00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:08.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.763+0000 7f0fe9400700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 0x7f0fd00a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009c60 tx=0x7f0fd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:08.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:08.767+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0fec068fc0 con 0x7f0fec102350 2024-03-16T10:05:08.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:08 smithi194 ceph-mon[36445]: purged_snaps scrub starts 2024-03-16T10:05:08.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:08 smithi194 ceph-mon[36445]: purged_snaps scrub ok 2024-03-16T10:05:08.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:08 smithi194 ceph-mon[36445]: osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] boot 2024-03-16T10:05:08.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:08 smithi194 ceph-mon[36445]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T10:05:08.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:08 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:09.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.002+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0fec1a3530 con 0x7f0fec102350 2024-03-16T10:05:09.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.003+0000 7f0fe2a00700 1 -- 172.21.15.39:0/943723884 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f0fdc013070 con 0x7f0fec102350 2024-03-16T10:05:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 msgr2=0x7f0fd00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 0x7f0fd00a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009c60 tx=0x7f0fd8009260 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 msgr2=0x7f0fec19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00bee0 tx=0x7f0fdc00bf10 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 shutdown_connections 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0fd00a2890 0x7f0fd00a4d50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fec103ef0 0x7f0fec1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fec102350 0x7f0fec19a7b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 --2- 172.21.15.39:0/943723884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fec103550 0x7f0fec19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 >> 172.21.15.39:0/943723884 conn(0x7f0fec0fd8f0 msgr2=0x7f0fec074500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:09.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:08 smithi039 ceph-mon[31510]: purged_snaps scrub starts 2024-03-16T10:05:09.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:08 smithi039 ceph-mon[31510]: purged_snaps scrub ok 2024-03-16T10:05:09.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:08 smithi039 ceph-mon[31510]: osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] boot 2024-03-16T10:05:09.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:08 smithi039 ceph-mon[31510]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T10:05:09.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:08 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:09.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.006+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 shutdown_connections 2024-03-16T10:05:09.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:09.007+0000 7f0ff0a00700 1 -- 172.21.15.39:0/943723884 wait complete. 2024-03-16T10:05:09.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:08 smithi105 ceph-mon[36559]: purged_snaps scrub starts 2024-03-16T10:05:09.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:08 smithi105 ceph-mon[36559]: purged_snaps scrub ok 2024-03-16T10:05:09.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:08 smithi105 ceph-mon[36559]: osd.11 [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] boot 2024-03-16T10:05:09.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:08 smithi105 ceph-mon[36559]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T10:05:09.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:08 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:05:09.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1710583507,"num_in_osds":12,"osd_in_since":1710583488,"num_remapped_pgs":0} 2024-03-16T10:05:09.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:05:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T10:05:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/943723884' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T10:05:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: Detected new or changed devices on smithi194 2024-03-16T10:05:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:10.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:09 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/943723884' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: Detected new or changed devices on smithi194 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:10.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:10.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:09 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T10:05:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T10:05:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/943723884' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T10:05:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: Detected new or changed devices on smithi194 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:10.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:09 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:10.293 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:05:12.156 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:11 smithi039 ceph-mon[31510]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:11 smithi194 ceph-mon[36445]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 -- 172.21.15.39:0/3821697242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763406b130 msgr2=0x7f763410e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 --2- 172.21.15.39:0/3821697242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763406b130 0x7f763410e870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f761c009990 tx=0x7f761c009ca0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 -- 172.21.15.39:0/3821697242 shutdown_connections 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 --2- 172.21.15.39:0/3821697242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763410edb0 0x7f76341111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 --2- 172.21.15.39:0/3821697242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406a810 0x7f763406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 --2- 172.21.15.39:0/3821697242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763406b130 0x7f763410e870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 -- 172.21.15.39:0/3821697242 >> 172.21.15.39:0/3821697242 conn(0x7f7634077220 msgr2=0x7f7634077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 -- 172.21.15.39:0/3821697242 shutdown_connections 2024-03-16T10:05:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.324+0000 7f763b600700 1 -- 172.21.15.39:0/3821697242 wait complete. 2024-03-16T10:05:12.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.325+0000 7f763b600700 1 Processor -- start 2024-03-16T10:05:12.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.325+0000 7f763b600700 1 -- start start 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.325+0000 7f763b600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 0x7f76341a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f763b600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406b130 0x7f76341a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f763b600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 0x7f763419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f763b600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7634114800 con 0x7f763410edb0 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f763b600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7634114680 con 0x7f763406b130 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f763b600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7634114980 con 0x7f763406a810 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7639600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 0x7f763419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 0x7f76341a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7639600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 0x7f763419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59976/0 (socket says 172.21.15.39:59976) 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7639600700 1 -- 172.21.15.39:0/3401705620 learned_addr learned my addr 172.21.15.39:0/3401705620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f762be00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406b130 0x7f76341a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 -- 172.21.15.39:0/3401705620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406b130 msgr2=0x7f76341a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406b130 0x7f76341a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 -- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 msgr2=0x7f763419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 0x7f763419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.326+0000 7f7638c00700 1 -- 172.21.15.39:0/3401705620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763419b260 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f7638c00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 0x7f76341a08f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9f0 tx=0x7f762400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400c9c0 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762400cb60 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624018680 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763419b550 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.327+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a77c0 con 0x7f763406a810 2024-03-16T10:05:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.328+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7634068f90 con 0x7f763406a810 2024-03-16T10:05:12.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.330+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7624010430 con 0x7f763406a810 2024-03-16T10:05:12.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.332+0000 7f7632000700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 0x7f76180a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:12.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.332+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f76240ce720 con 0x7f763406a810 2024-03-16T10:05:12.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.332+0000 7f762be00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 0x7f76180a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:12.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.333+0000 7f762be00700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 0x7f76180a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7634102710 tx=0x7f761c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:12.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.336+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f762409bbc0 con 0x7f763406a810 2024-03-16T10:05:12.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:11 smithi105 ceph-mon[36559]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:12.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.564+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f763402db70 con 0x7f763406a810 2024-03-16T10:05:12.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.566+0000 7f7632000700 1 -- 172.21.15.39:0/3401705620 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18445 (secure 0 0 0) 0x7f762409b4c0 con 0x7f763406a810 2024-03-16T10:05:12.567 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:12.567 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:05:08.845239+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:05:12.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.569+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 msgr2=0x7f76180a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.569+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 0x7f76180a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7634102710 tx=0x7f761c005fd0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 msgr2=0x7f76341a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 0x7f76341a08f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f762400e9f0 tx=0x7f762400ed00 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 shutdown_connections 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f76180a2910 0x7f76180a4dd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f763410edb0 0x7f763419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f763406a810 0x7f76341a08f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 --2- 172.21.15.39:0/3401705620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f763406b130 0x7f76341a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 >> 172.21.15.39:0/3401705620 conn(0x7f7634077220 msgr2=0x7f7634110220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 shutdown_connections 2024-03-16T10:05:12.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:12.570+0000 7f763b600700 1 -- 172.21.15.39:0/3401705620 wait complete. 2024-03-16T10:05:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:12 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3401705620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:12 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3401705620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:13.309 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:05:13.310 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-16T10:05:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:12 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3401705620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:05:14.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:13 smithi039 ceph-mon[31510]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:13 smithi194 ceph-mon[36445]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:13 smithi105 ceph-mon[36559]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 -- 172.21.15.39:0/2645740975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 msgr2=0x7f699810a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 --2- 172.21.15.39:0/2645740975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699810a970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f69880099c0 tx=0x7f6988009cd0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 -- 172.21.15.39:0/2645740975 shutdown_connections 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 --2- 172.21.15.39:0/2645740975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 0x7f6998111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 --2- 172.21.15.39:0/2645740975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699810a970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 --2- 172.21.15.39:0/2645740975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f6998104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 -- 172.21.15.39:0/2645740975 >> 172.21.15.39:0/2645740975 conn(0x7f69980fffb0 msgr2=0x7f69981023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:15.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 -- 172.21.15.39:0/2645740975 shutdown_connections 2024-03-16T10:05:15.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.839+0000 7f699d200700 1 -- 172.21.15.39:0/2645740975 wait complete. 2024-03-16T10:05:15.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.840+0000 7f699d200700 1 Processor -- start 2024-03-16T10:05:15.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 -- start start 2024-03-16T10:05:15.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 0x7f69981a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6998114820 con 0x7f699810a590 2024-03-16T10:05:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69981146a0 con 0x7f69981044e0 2024-03-16T10:05:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f699d200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f69981149a0 con 0x7f6998104ea0 2024-03-16T10:05:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6996800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6995e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6996800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59996/0 (socket says 172.21.15.39:59996) 2024-03-16T10:05:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6996800700 1 -- 172.21.15.39:0/1203938087 learned_addr learned my addr 172.21.15.39:0/1203938087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6995e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59330/0 (socket says 172.21.15.39:59330) 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.841+0000 7f6995400700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 0x7f69981a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 -- 172.21.15.39:0/1203938087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 msgr2=0x7f69981a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 0x7f69981a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 -- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 msgr2=0x7f699819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 -- 172.21.15.39:0/1203938087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699819b120 con 0x7f69981044e0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f6995e00700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69880098e0 tx=0x7f6988003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.842+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988009e60 con 0x7f69981044e0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.843+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6988003680 con 0x7f69981044e0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.843+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699819b3b0 con 0x7f69981044e0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.843+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800fd40 con 0x7f69981044e0 2024-03-16T10:05:15.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.843+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981a76d0 con 0x7f69981044e0 2024-03-16T10:05:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.844+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6998068f90 con 0x7f69981044e0 2024-03-16T10:05:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.846+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f698800f460 con 0x7f69981044e0 2024-03-16T10:05:15.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.847+0000 7f698ea00700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 0x7f697c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:15.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.847+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f69880cf6b0 con 0x7f69981044e0 2024-03-16T10:05:15.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.848+0000 7f6995400700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 0x7f697c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:15.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.848+0000 7f6995400700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 0x7f697c0a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6984009680 tx=0x7f6984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:15.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:15.850+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6988013070 con 0x7f69981044e0 2024-03-16T10:05:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:15 smithi039 ceph-mon[31510]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:16.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.074+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69981a78a0 con 0x7f69981044e0 2024-03-16T10:05:16.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.075+0000 7f698ea00700 1 -- 172.21.15.39:0/1203938087 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f698809cbe0 con 0x7f69981044e0 2024-03-16T10:05:16.076 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 msgr2=0x7f697c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 0x7f697c0a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6984009680 tx=0x7f6984009260 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 msgr2=0x7f69981a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69880098e0 tx=0x7f6988003e00 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 shutdown_connections 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f697c0a2910 0x7f697c0a4dd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f699810a590 0x7f699819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69981044e0 0x7f69981a0800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 --2- 172.21.15.39:0/1203938087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6998104ea0 0x7f69981a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 >> 172.21.15.39:0/1203938087 conn(0x7f69980fffb0 msgr2=0x7f69981014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 shutdown_connections 2024-03-16T10:05:16.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:16.078+0000 7f699d200700 1 -- 172.21.15.39:0/1203938087 wait complete. 2024-03-16T10:05:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:15 smithi194 ceph-mon[36445]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:15 smithi105 ceph-mon[36559]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.a on smithi039 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.d on smithi039 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.g on smithi039 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.j on smithi039 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.b on smithi105 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.e on smithi105 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.h on smithi105 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.k on smithi105 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.c on smithi194 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.f on smithi194 2024-03-16T10:05:16.899 INFO:tasks.cephadm:Adding mds.i on smithi194 2024-03-16T10:05:16.900 INFO:tasks.cephadm:Adding mds.l on smithi194 2024-03-16T10:05:16.900 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l' 2024-03-16T10:05:17.150 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:05:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:16 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1203938087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:17.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:16 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1203938087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:17.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:16 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1203938087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:18.179 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:17 smithi194 ceph-mon[36445]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:18.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:17 smithi039 ceph-mon[31510]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:18.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:17 smithi105 ceph-mon[36559]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.702+0000 7ff8d8876700 1 -- 172.21.15.194:0/2956966737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102470 msgr2=0x7ff8d41028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.702+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2956966737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102470 0x7ff8d41028d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009990 tx=0x7ff8c8009ca0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 -- 172.21.15.194:0/2956966737 shutdown_connections 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2956966737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102e10 0x7ff8d41096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2956966737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4101270 0x7ff8d4101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.705 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2956966737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102470 0x7ff8d41028d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 -- 172.21.15.194:0/2956966737 >> 172.21.15.194:0/2956966737 conn(0x7ff8d40fc830 msgr2=0x7ff8d40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 -- 172.21.15.194:0/2956966737 shutdown_connections 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 -- 172.21.15.194:0/2956966737 wait complete. 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.703+0000 7ff8d8876700 1 Processor -- start 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 -- start start 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4101270 0x7ff8d406b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4102470 0x7ff8d419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410ce50 con 0x7ff8d4101270 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410ccd0 con 0x7ff8d4102470 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d8876700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410cfd0 con 0x7ff8d4102e10 2024-03-16T10:05:18.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:33452/0 (socket says 172.21.15.194:33452) 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 -- 172.21.15.194:0/2491150803 learned_addr learned my addr 172.21.15.194:0/2491150803 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d1d9b700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4102470 0x7ff8d419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 -- 172.21.15.194:0/2491150803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4102470 msgr2=0x7ff8d419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4102470 0x7ff8d419d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 -- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4101270 msgr2=0x7ff8d406b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4101270 0x7ff8d406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 -- 172.21.15.194:0/2491150803 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d41a0140 con 0x7ff8d4102e10 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8d2d9d700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00b970 tx=0x7ff8cc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.704+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc019070 con 0x7ff8d4102e10 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.705+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8cc00ce10 con 0x7ff8d4102e10 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.705+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d41a0430 con 0x7ff8d4102e10 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.705+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc007370 con 0x7ff8d4102e10 2024-03-16T10:05:18.707 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.705+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d41a09a0 con 0x7ff8d4102e10 2024-03-16T10:05:18.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.705+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d4068f90 con 0x7ff8d4102e10 2024-03-16T10:05:18.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.707+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff8cc01c070 con 0x7ff8d4102e10 2024-03-16T10:05:18.708 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.707+0000 7ff8c37fe700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 0x7ff8bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:18.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.708+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8cc0cde10 con 0x7ff8d4102e10 2024-03-16T10:05:18.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.708+0000 7ff8d259c700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 0x7ff8bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:18.709 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.709+0000 7ff8d259c700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 0x7ff8bc0a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8d41022d0 tx=0x7ff8c4005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:18.711 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.711+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8cc09b350 con 0x7ff8d4102e10 2024-03-16T10:05:18.915 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.915+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}) -- 0x7ff8d4062910 con 0x7ff8bc0a2890 2024-03-16T10:05:18.931 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.931+0000 7ff8c37fe700 1 -- 172.21.15.194:0/2491150803 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff8d4062910 con 0x7ff8bc0a2890 2024-03-16T10:05:18.932 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mds.all update... 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.933+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 msgr2=0x7ff8bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.933+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 0x7ff8bc0a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8d41022d0 tx=0x7ff8c4005d20 comp rx=0 tx=0).stop 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.933+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 msgr2=0x7ff8d419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.933+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00b970 tx=0x7ff8cc00bc80 comp rx=0 tx=0).stop 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 shutdown_connections 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff8bc0a2890 0x7ff8bc0a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8d4102e10 0x7ff8d419f870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4101270 0x7ff8d406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 --2- 172.21.15.194:0/2491150803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8d4102470 0x7ff8d419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:18.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 >> 172.21.15.194:0/2491150803 conn(0x7ff8d40fc830 msgr2=0x7ff8d4105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:18.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 shutdown_connections 2024-03-16T10:05:18.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T10:05:18.934+0000 7ff8d8876700 1 -- 172.21.15.194:0/2491150803 wait complete. 2024-03-16T10:05:19.698 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.a.service 2024-03-16T10:05:19.707 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.d.service 2024-03-16T10:05:19.753 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.g.service 2024-03-16T10:05:19.798 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.j.service 2024-03-16T10:05:19.844 DEBUG:teuthology.orchestra.run.smithi105:mds.b> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.b.service 2024-03-16T10:05:19.848 DEBUG:teuthology.orchestra.run.smithi105:mds.e> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.e.service 2024-03-16T10:05:19.891 DEBUG:teuthology.orchestra.run.smithi105:mds.h> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.h.service 2024-03-16T10:05:19.933 DEBUG:teuthology.orchestra.run.smithi105:mds.k> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.k.service 2024-03-16T10:05:19.976 DEBUG:teuthology.orchestra.run.smithi194:mds.c> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.c.service 2024-03-16T10:05:19.979 DEBUG:teuthology.orchestra.run.smithi194:mds.f> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.f.service 2024-03-16T10:05:20.022 DEBUG:teuthology.orchestra.run.smithi194:mds.i> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.i.service 2024-03-16T10:05:20.065 DEBUG:teuthology.orchestra.run.smithi194:mds.l> sudo journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.l.service 2024-03-16T10:05:20.108 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-16T10:05:20.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:05:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:19 smithi039 ceph-mon[31510]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:19 smithi039 ceph-mon[31510]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:05:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:19 smithi039 ceph-mon[31510]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T10:05:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:20.188 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:19 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:20.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:19 smithi194 ceph-mon[36445]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:20.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:19 smithi194 ceph-mon[36445]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:05:20.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:19 smithi194 ceph-mon[36445]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T10:05:20.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:20.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:19 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 -- 172.21.15.39:0/1277499539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9901055b0 msgr2=0x7fc99010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 --2- 172.21.15.39:0/1277499539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9901055b0 0x7fc99010be40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ca0 tx=0x7fc98802c0f0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 -- 172.21.15.39:0/1277499539 shutdown_connections 2024-03-16T10:05:20.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 --2- 172.21.15.39:0/1277499539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9901055b0 0x7fc99010be40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 --2- 172.21.15.39:0/1277499539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 0x7fc990105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 --2- 172.21.15.39:0/1277499539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc990103a00 0x7fc990103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.337+0000 7fc995a00640 1 -- 172.21.15.39:0/1277499539 >> 172.21.15.39:0/1277499539 conn(0x7fc9900fef90 msgr2=0x7fc9901013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.338+0000 7fc995a00640 1 -- 172.21.15.39:0/1277499539 shutdown_connections 2024-03-16T10:05:20.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.338+0000 7fc995a00640 1 -- 172.21.15.39:0/1277499539 wait complete. 2024-03-16T10:05:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.339+0000 7fc995a00640 1 Processor -- start 2024-03-16T10:05:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.339+0000 7fc995a00640 1 -- start start 2024-03-16T10:05:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 0x7fc99019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 0x7fc9901a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010f680 con 0x7fc9901055b0 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010f500 con 0x7fc990104bf0 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc995a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010f800 con 0x7fc990103a00 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41880/0 (socket says 172.21.15.39:41880) 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 -- 172.21.15.39:0/2130464434 learned_addr learned my addr 172.21.15.39:0/2130464434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98e000640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 0x7fc99019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98f400640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 0x7fc9901a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 -- 172.21.15.39:0/2130464434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 msgr2=0x7fc99019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 0x7fc99019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 -- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 msgr2=0x7fc9901a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 0x7fc9901a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98ea00640 1 -- 172.21.15.39:0/2130464434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9901a18d0 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.340+0000 7fc98f400640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 0x7fc9901a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc98ea00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9c0 tx=0x7fc98000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000cd80 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9800035f0 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980010690 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901a1bc0 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.341+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a20c0 con 0x7fc990103a00 2024-03-16T10:05:20.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.342+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99010ad20 con 0x7fc990103a00 2024-03-16T10:05:20.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.344+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc9800037a0 con 0x7fc990103a00 2024-03-16T10:05:20.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.345+0000 7fc987400640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 0x7fc9740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.346+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9800cea40 con 0x7fc990103a00 2024-03-16T10:05:20.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.346+0000 7fc98e000640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 0x7fc9740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.346+0000 7fc98e000640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 0x7fc9740a4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc99019b190 tx=0x7fc978005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:20.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.350+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc98009bee0 con 0x7fc990103a00 2024-03-16T10:05:20.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:19 smithi105 ceph-mon[36559]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:19 smithi105 ceph-mon[36559]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:05:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:19 smithi105 ceph-mon[36559]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T10:05:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:19 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:05:20.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.554+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9900624e0 con 0x7fc990103a00 2024-03-16T10:05:20.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.555+0000 7fc987400640 1 -- 172.21.15.39:0/2130464434 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18445 (secure 0 0 0) 0x7fc98009b7e0 con 0x7fc990103a00 2024-03-16T10:05:20.557 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:20.557 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:05:08.845239+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 msgr2=0x7fc9740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 0x7fc9740a4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc99019b190 tx=0x7fc978005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 msgr2=0x7fc990199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9c0 tx=0x7fc98000ee80 comp rx=0 tx=0).stop 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 shutdown_connections 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.558+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9740a2880 0x7fc9740a4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9901055b0 0x7fc9901a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc990104bf0 0x7fc99019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 --2- 172.21.15.39:0/2130464434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc990103a00 0x7fc990199c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 >> 172.21.15.39:0/2130464434 conn(0x7fc9900fef90 msgr2=0x7fc990107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:20.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 shutdown_connections 2024-03-16T10:05:20.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.559+0000 7fc995a00640 1 -- 172.21.15.39:0/2130464434 wait complete. 2024-03-16T10:05:20.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:05:20.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:05:20.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 -- 172.21.15.39:0/599863892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec1055b0 msgr2=0x7f2fec10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 --2- 172.21.15.39:0/599863892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec1055b0 0x7f2fec10be40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009860 tx=0x7f2fe402bb90 comp rx=0 tx=0).stop 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 -- 172.21.15.39:0/599863892 shutdown_connections 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 --2- 172.21.15.39:0/599863892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec1055b0 0x7f2fec10be40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 --2- 172.21.15.39:0/599863892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec104bf0 0x7f2fec105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 --2- 172.21.15.39:0/599863892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec103a00 0x7f2fec103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 -- 172.21.15.39:0/599863892 >> 172.21.15.39:0/599863892 conn(0x7f2fec0fef90 msgr2=0x7f2fec1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 -- 172.21.15.39:0/599863892 shutdown_connections 2024-03-16T10:05:20.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.808+0000 7f2ff3000640 1 -- 172.21.15.39:0/599863892 wait complete. 2024-03-16T10:05:20.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 Processor -- start 2024-03-16T10:05:20.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 -- start start 2024-03-16T10:05:20.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec104bf0 0x7f2fec19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec1055b0 0x7f2fec1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10f680 con 0x7f2fec103a00 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10f500 con 0x7f2fec1055b0 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2ff3000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10f800 con 0x7f2fec104bf0 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.809+0000 7f2febe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37680/0 (socket says 172.21.15.39:37680) 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 -- 172.21.15.39:0/3505168545 learned_addr learned my addr 172.21.15.39:0/3505168545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2feb400640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec104bf0 0x7f2fec19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2ff1000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec1055b0 0x7f2fec1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 -- 172.21.15.39:0/3505168545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec104bf0 msgr2=0x7f2fec19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec104bf0 0x7f2fec19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 -- 172.21.15.39:0/3505168545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec1055b0 msgr2=0x7f2fec1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec1055b0 0x7f2fec1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 -- 172.21.15.39:0/3505168545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec1a18d0 con 0x7f2fec103a00 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.810+0000 7f2febe00640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400e9f0 tx=0x7f2fd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.811+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ccc0 con 0x7f2fec103a00 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.811+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd40035f0 con 0x7f2fec103a00 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.811+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec1a1bc0 con 0x7f2fec103a00 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.811+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ce60 con 0x7f2fec103a00 2024-03-16T10:05:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.811+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec1a20c0 con 0x7f2fec103a00 2024-03-16T10:05:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.812+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec10ad20 con 0x7f2fec103a00 2024-03-16T10:05:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.815+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2fd4003b10 con 0x7f2fec103a00 2024-03-16T10:05:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.816+0000 7f2fe8c00640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 0x7f2fd00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.816+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2fd40ce910 con 0x7f2fec103a00 2024-03-16T10:05:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.816+0000 7f2feb400640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 0x7f2fd00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.817+0000 7f2feb400640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 0x7f2fd00a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fec19b190 tx=0x7f2fdc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:20.821+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2fd409bdb0 con 0x7f2fec103a00 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.008 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:20 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2130464434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:21.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.026+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fec0623e0 con 0x7f2fec103a00 2024-03-16T10:05:21.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.027+0000 7f2fe8c00640 1 -- 172.21.15.39:0/3505168545 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f2fd40bb030 con 0x7f2fec103a00 2024-03-16T10:05:21.028 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:05:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 msgr2=0x7f2fd00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 0x7f2fd00a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fec19b190 tx=0x7f2fdc009500 comp rx=0 tx=0).stop 2024-03-16T10:05:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 msgr2=0x7f2fec199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400e9f0 tx=0x7f2fd400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 shutdown_connections 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2fd00a2800 0x7f2fd00a4cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fec1055b0 0x7f2fec1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fec104bf0 0x7f2fec19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 --2- 172.21.15.39:0/3505168545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fec103a00 0x7f2fec199c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 >> 172.21.15.39:0/3505168545 conn(0x7f2fec0fef90 msgr2=0x7f2fec107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 shutdown_connections 2024-03-16T10:05:21.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.030+0000 7f2ff3000640 1 -- 172.21.15.39:0/3505168545 wait complete. 2024-03-16T10:05:21.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:21.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:21.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:21.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:20 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2130464434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:21.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 -- 172.21.15.39:0/841258265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c107e00 msgr2=0x7f3d4c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 --2- 172.21.15.39:0/841258265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c107e00 0x7f3d4c108220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c009900 tx=0x7f3d3c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 -- 172.21.15.39:0/841258265 shutdown_connections 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 --2- 172.21.15.39:0/841258265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 --2- 172.21.15.39:0/841258265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 --2- 172.21.15.39:0/841258265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c107e00 0x7f3d4c108220 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 -- 172.21.15.39:0/841258265 >> 172.21.15.39:0/841258265 conn(0x7f3d4c076dc0 msgr2=0x7f3d4c0791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 -- 172.21.15.39:0/841258265 shutdown_connections 2024-03-16T10:05:21.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.275+0000 7f3d51600640 1 -- 172.21.15.39:0/841258265 wait complete. 2024-03-16T10:05:21.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.276+0000 7f3d51600640 1 Processor -- start 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.276+0000 7f3d51600640 1 -- start start 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c107e00 0x7f3d4c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c113ac0 con 0x7f3d4c1099b0 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c113940 con 0x7f3d4c107e00 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d51600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c113c40 con 0x7f3d4c108ff0 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d4b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c1a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37704/0 (socket says 172.21.15.39:37704) 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d4b400640 1 -- 172.21.15.39:0/86929743 learned_addr learned my addr 172.21.15.39:0/86929743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d4aa00640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c107e00 0x7f3d4c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.277+0000 7f3d42000640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 -- 172.21.15.39:0/86929743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c107e00 msgr2=0x7f3d4c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c107e00 0x7f3d4c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 -- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 msgr2=0x7f3d4c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 -- 172.21.15.39:0/86929743 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c1a5d30 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d42000640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c19e650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400e9f0 tx=0x7f3d3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400ccc0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d34005020 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d340049e0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c1a5fc0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.278+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c1a6490 con 0x7f3d4c108ff0 2024-03-16T10:05:21.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.280+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4c10f0c0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.281+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3d34005560 con 0x7f3d4c108ff0 2024-03-16T10:05:21.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.283+0000 7f3d43e00640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 0x7f3d300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:21.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.283+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3d340ce620 con 0x7f3d4c108ff0 2024-03-16T10:05:21.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.283+0000 7f3d4aa00640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 0x7f3d300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:21.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.284+0000 7f3d4aa00640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 0x7f3d300a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c02c170 tx=0x7f3d3c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:21.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.287+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3d340051d0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:21.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:20 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2130464434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:05:21.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.532+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d4c062480 con 0x7f3d4c108ff0 2024-03-16T10:05:21.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.533+0000 7f3d43e00640 1 -- 172.21.15.39:0/86929743 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f3d3409bac0 con 0x7f3d4c108ff0 2024-03-16T10:05:21.534 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:21.534 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-16T10:05:21.534 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 1 2024-03-16T10:05:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.535+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 msgr2=0x7f3d300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 0x7f3d300a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c02c170 tx=0x7f3d3c02df90 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 msgr2=0x7f3d4c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c19e650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400e9f0 tx=0x7f3d3400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 shutdown_connections 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3d300a2800 0x7f3d300a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d4c1099b0 0x7f3d4c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d4c108ff0 0x7f3d4c19e650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 --2- 172.21.15.39:0/86929743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d4c107e00 0x7f3d4c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 >> 172.21.15.39:0/86929743 conn(0x7f3d4c076dc0 msgr2=0x7f3d4c10c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 shutdown_connections 2024-03-16T10:05:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:21.536+0000 7f3d51600640 1 -- 172.21.15.39:0/86929743 wait complete. 2024-03-16T10:05:22.038 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:21 smithi194 ceph-mon[36445]: Deploying daemon mds.l on smithi194 2024-03-16T10:05:22.039 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:21 smithi194 ceph-mon[36445]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:22.039 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:21 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3505168545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:22.039 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:21 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/86929743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:21 smithi039 ceph-mon[31510]: Deploying daemon mds.l on smithi194 2024-03-16T10:05:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:21 smithi039 ceph-mon[31510]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:22.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:21 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3505168545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:22.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:21 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/86929743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:22.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:21 smithi105 ceph-mon[36559]: Deploying daemon mds.l on smithi194 2024-03-16T10:05:22.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:21 smithi105 ceph-mon[36559]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:22.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:21 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3505168545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:05:22.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:21 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/86929743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:22.441 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 systemd[1]: Starting Ceph mds.l for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:22.706 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 podman[67048]: 2024-03-16 10:05:22.343947816 +0000 UTC m=+0.018193334 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:22.706 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 podman[67048]: 2024-03-16 10:05:22.463310163 +0000 UTC m=+0.137555660 container create 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T10:05:23.206 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 podman[67048]: 2024-03-16 10:05:22.725146902 +0000 UTC m=+0.399392427 container init 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T10:05:23.207 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 podman[67048]: 2024-03-16 10:05:22.727962834 +0000 UTC m=+0.402208342 container start 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T10:05:23.207 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 ceph-mds[67067]: -- 172.21.15.194:0/4190988591 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560a7095a40 con 0x5560a7069000 2024-03-16T10:05:23.207 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: starting mds.l at 2024-03-16T10:05:23.207 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 bash[67048]: 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b 2024-03-16T10:05:23.207 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:05:22 smithi194 systemd[1]: Started Ceph mds.l for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:23.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 -- 172.21.15.39:0/656131907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0104bb0 msgr2=0x7fecb0106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 --2- 172.21.15.39:0/656131907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0104bb0 0x7fecb0106fa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feca0009860 tx=0x7feca002bb70 comp rx=0 tx=0).stop 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 -- 172.21.15.39:0/656131907 shutdown_connections 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 --2- 172.21.15.39:0/656131907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb01074e0 0x7fecb0109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 --2- 172.21.15.39:0/656131907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb0104bb0 0x7fecb0106fa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 --2- 172.21.15.39:0/656131907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:23.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 -- 172.21.15.39:0/656131907 >> 172.21.15.39:0/656131907 conn(0x7fecb00fbc70 msgr2=0x7fecb00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 -- 172.21.15.39:0/656131907 shutdown_connections 2024-03-16T10:05:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.796+0000 7fecb8800640 1 -- 172.21.15.39:0/656131907 wait complete. 2024-03-16T10:05:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.797+0000 7fecb8800640 1 Processor -- start 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.797+0000 7fecb8800640 1 -- start start 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb0104bb0 0x7fecb019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 0x7fecb01a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d470 con 0x7fecb0102280 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d2f0 con 0x7fecb01074e0 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb8800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010d5f0 con 0x7fecb0104bb0 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37720/0 (socket says 172.21.15.39:37720) 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 -- 172.21.15.39:0/2508921027 learned_addr learned my addr 172.21.15.39:0/2508921027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb6800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 0x7fecb01a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5400640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb0104bb0 0x7fecb019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 -- 172.21.15.39:0/2508921027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb0104bb0 msgr2=0x7fecb019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb0104bb0 0x7fecb019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 -- 172.21.15.39:0/2508921027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 msgr2=0x7fecb01a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 0x7fecb01a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb5e00640 1 -- 172.21.15.39:0/2508921027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01a1950 con 0x7fecb0102280 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.798+0000 7fecb6800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 0x7fecb01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecb5e00640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feca400e9c0 tx=0x7feca400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca400cde0 con 0x7fecb0102280 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca40035f0 con 0x7fecb0102280 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb01a1c40 con 0x7fecb0102280 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4010690 con 0x7fecb0102280 2024-03-16T10:05:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.799+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb01a2190 con 0x7fecb0102280 2024-03-16T10:05:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.800+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb0109030 con 0x7fecb0102280 2024-03-16T10:05:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.802+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7feca40037a0 con 0x7fecb0102280 2024-03-16T10:05:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.805+0000 7fecaaa00640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 0x7fec980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.805+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feca40cea20 con 0x7fecb0102280 2024-03-16T10:05:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.805+0000 7fecb5400640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 0x7fec980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.805+0000 7fecb5400640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 0x7fec980a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feca00349e0 tx=0x7feca0034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:23.809+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feca409bec0 con 0x7fecb0102280 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6832/1279561047,v1:172.21.15.194:6833/1279561047] up:boot 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: fsmap 1 up:standby 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:23.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:23 smithi105 ceph-mon[36559]: Deploying daemon mds.k on smithi105 2024-03-16T10:05:24.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6832/1279561047,v1:172.21.15.194:6833/1279561047] up:boot 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: fsmap 1 up:standby 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:24.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:23 smithi039 ceph-mon[31510]: Deploying daemon mds.k on smithi105 2024-03-16T10:05:24.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.056+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecb002d1b0 con 0x7fecb0102280 2024-03-16T10:05:24.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.057+0000 7fecaaa00640 1 -- 172.21.15.39:0/2508921027 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7feca409b7c0 con 0x7fecb0102280 2024-03-16T10:05:24.057 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:24.058 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T10:05:24.058 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-03-16T10:05:24.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 msgr2=0x7fec980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 0x7fec980a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feca00349e0 tx=0x7feca0034ab0 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 msgr2=0x7fecb0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feca400e9c0 tx=0x7feca400ee80 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 shutdown_connections 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fec980a2800 0x7fec980a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01074e0 0x7fecb01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fecb0104bb0 0x7fecb019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 --2- 172.21.15.39:0/2508921027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecb0102280 0x7fecb0199d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 >> 172.21.15.39:0/2508921027 conn(0x7fecb00fbc70 msgr2=0x7fecb01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 shutdown_connections 2024-03-16T10:05:24.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:24.060+0000 7fecb8800640 1 -- 172.21.15.39:0/2508921027 wait complete. 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6832/1279561047,v1:172.21.15.194:6833/1279561047] up:boot 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: fsmap 1 up:standby 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:24.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:24.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:24.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:24.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:23 smithi194 ceph-mon[36445]: Deploying daemon mds.k on smithi105 2024-03-16T10:05:24.977 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:24 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2508921027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:25.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:24 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2508921027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:25.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:24 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2508921027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:25.365 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 systemd[1]: Starting Ceph mds.k for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:25.643 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 podman[66261]: 2024-03-16 10:05:25.278384302 +0000 UTC m=+0.017305443 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:25.643 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 podman[66261]: 2024-03-16 10:05:25.389458166 +0000 UTC m=+0.128379340 container create a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-16T10:05:25.644 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 podman[66261]: 2024-03-16 10:05:25.617496985 +0000 UTC m=+0.356418124 container init a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T10:05:25.644 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 podman[66261]: 2024-03-16 10:05:25.620004429 +0000 UTC m=+0.358925552 container start a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-16T10:05:25.644 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 ceph-mds[66281]: -- 172.21.15.105:0/1196404120 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557420fe5a40 con 0x557420fb8c00 2024-03-16T10:05:26.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:25 smithi105 ceph-mon[36559]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:25 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k[66277]: starting mds.k at 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 bash[66261]: a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb 2024-03-16T10:05:26.116 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:05:25 smithi105 systemd[1]: Started Ceph mds.k for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:26.166 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:25 smithi039 ceph-mon[31510]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:26.166 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.167 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:25 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:25 smithi194 ceph-mon[36445]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:25 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 -- 172.21.15.39:0/1575418045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a41000f0 msgr2=0x7f97a4100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:26.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 --2- 172.21.15.39:0/1575418045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a41000f0 0x7f97a4100510 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9794009900 tx=0x7f979402bcc0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 -- 172.21.15.39:0/1575418045 shutdown_connections 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 --2- 172.21.15.39:0/1575418045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41013f0 0x7f97a4101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 --2- 172.21.15.39:0/1575418045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a4100a50 0x7f97a4100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 --2- 172.21.15.39:0/1575418045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a41000f0 0x7f97a4100510 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 -- 172.21.15.39:0/1575418045 >> 172.21.15.39:0/1575418045 conn(0x7f97a40fbc70 msgr2=0x7f97a40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 -- 172.21.15.39:0/1575418045 shutdown_connections 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.302+0000 7f97aa200640 1 -- 172.21.15.39:0/1575418045 wait complete. 2024-03-16T10:05:26.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.303+0000 7f97aa200640 1 Processor -- start 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 -- start start 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41000f0 0x7f97a4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 0x7f97a41a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410f6a0 con 0x7f97a4100a50 2024-03-16T10:05:26.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410f520 con 0x7f97a41013f0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97aa200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97a410f820 con 0x7f97a41000f0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37738/0 (socket says 172.21.15.39:37738) 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97a2a00640 1 -- 172.21.15.39:0/3433250390 learned_addr learned my addr 172.21.15.39:0/3433250390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97a3e00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 0x7f97a41a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.304+0000 7f97a3400640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41000f0 0x7f97a4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 -- 172.21.15.39:0/3433250390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41000f0 msgr2=0x7f97a4199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41000f0 0x7f97a4199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 -- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 msgr2=0x7f97a419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 -- 172.21.15.39:0/3433250390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a41a1910 con 0x7f97a41013f0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a2a00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97a3e00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 0x7f97a41a1270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f979c009fd0 tx=0x7f979c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:26.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c00ce20 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c005020 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c010720 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.305+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41a1c00 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.306+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41a20d0 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.307+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a410afe0 con 0x7f97a41013f0 2024-03-16T10:05:26.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.308+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f979c005540 con 0x7f97a41013f0 2024-03-16T10:05:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.309+0000 7f979be00640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 0x7f97880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.310+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f979c0ced80 con 0x7f97a41013f0 2024-03-16T10:05:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.310+0000 7f97a3400640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 0x7f97880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:26.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.310+0000 7f97a3400640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 0x7f97880a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f979402c2d0 tx=0x7f979402c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:26.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.314+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f979c09c220 con 0x7f97a41013f0 2024-03-16T10:05:26.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.568+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97a40623e0 con 0x7f97a41013f0 2024-03-16T10:05:26.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.569+0000 7f979be00640 1 -- 172.21.15.39:0/3433250390 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f979c09bb20 con 0x7f97a41013f0 2024-03-16T10:05:26.570 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:26.570 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T10:05:26.570 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 msgr2=0x7f97880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 0x7f97880a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f979402c2d0 tx=0x7f979402c910 comp rx=0 tx=0).stop 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 msgr2=0x7f97a41a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 0x7f97a41a1270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f979c009fd0 tx=0x7f979c00ef10 comp rx=0 tx=0).stop 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 shutdown_connections 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f97880a2880 0x7f97880a4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a41013f0 0x7f97a41a1270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97a4100a50 0x7f97a419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 --2- 172.21.15.39:0/3433250390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97a41000f0 0x7f97a4199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 >> 172.21.15.39:0/3433250390 conn(0x7f97a40fbc70 msgr2=0x7f97a40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 shutdown_connections 2024-03-16T10:05:26.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:26.572+0000 7f97aa200640 1 -- 172.21.15.39:0/3433250390 wait complete. 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6832/2269577878,v1:172.21.15.105:6833/2269577878] up:boot 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: fsmap 2 up:standby 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:26.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: Deploying daemon mds.j on smithi039 2024-03-16T10:05:26.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3433250390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:27.107 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6832/2269577878,v1:172.21.15.105:6833/2269577878] up:boot 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: fsmap 2 up:standby 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:27.108 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: Deploying daemon mds.j on smithi039 2024-03-16T10:05:27.109 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3433250390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6832/2269577878,v1:172.21.15.105:6833/2269577878] up:boot 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: fsmap 2 up:standby 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:27.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: Deploying daemon mds.j on smithi039 2024-03-16T10:05:27.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3433250390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:27 smithi039 ceph-mon[31510]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:28.029 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:27 smithi105 ceph-mon[36559]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:28.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:27 smithi194 ceph-mon[36445]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:28.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 -- 172.21.15.39:0/232510415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81241009e0 msgr2=0x7f8124100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 --2- 172.21.15.39:0/232510415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81241009e0 0x7f8124100e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8110009900 tx=0x7f811002bc50 comp rx=0 tx=0).stop 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 -- 172.21.15.39:0/232510415 shutdown_connections 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 --2- 172.21.15.39:0/232510415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124101380 0x7f8124101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 --2- 172.21.15.39:0/232510415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81241009e0 0x7f8124100e40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:28.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 --2- 172.21.15.39:0/232510415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8124100080 0x7f81241004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:28.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 -- 172.21.15.39:0/232510415 >> 172.21.15.39:0/232510415 conn(0x7f81240fbc60 msgr2=0x7f81240fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:28.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 -- 172.21.15.39:0/232510415 shutdown_connections 2024-03-16T10:05:28.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.928+0000 7f812bc00640 1 -- 172.21.15.39:0/232510415 wait complete. 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.929+0000 7f812bc00640 1 Processor -- start 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 -- start start 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 0x7f8124199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81241009e0 0x7f812419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 0x7f81241a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f812410f650 con 0x7f8124100080 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f812410f4d0 con 0x7f8124101380 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f812bc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f812410f7d0 con 0x7f81241009e0 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f8129200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 0x7f8124199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f8129c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 0x7f81241a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f8129200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 0x7f8124199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37762/0 (socket says 172.21.15.39:37762) 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f8129200640 1 -- 172.21.15.39:0/1625255691 learned_addr learned my addr 172.21.15.39:0/1625255691 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.930+0000 7f8123e00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81241009e0 0x7f812419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 -- 172.21.15.39:0/1625255691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81241009e0 msgr2=0x7f812419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81241009e0 0x7f812419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 -- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 msgr2=0x7f8124199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 0x7f8124199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 -- 172.21.15.39:0/1625255691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241a1960 con 0x7f8124101380 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8129c00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 0x7f81241a12f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f811c009fd0 tx=0x7f811c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c00ce20 con 0x7f8124101380 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241a1c50 con 0x7f8124101380 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811c005020 con 0x7f8124101380 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c010720 con 0x7f8124101380 2024-03-16T10:05:28.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.931+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241a2120 con 0x7f8124101380 2024-03-16T10:05:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.932+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812410add0 con 0x7f8124101380 2024-03-16T10:05:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.934+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f811c005540 con 0x7f8124101380 2024-03-16T10:05:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.936+0000 7f8121600640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 0x7f810c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.936+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f811c0ceca0 con 0x7f8124101380 2024-03-16T10:05:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.936+0000 7f8129200640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 0x7f810c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.937+0000 7f8129200640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 0x7f810c0a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8118009c90 tx=0x7f8118005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:28.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:28.940+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f811c09c1e0 con 0x7f8124101380 2024-03-16T10:05:29.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.200+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f812402d3b0 con 0x7f8124101380 2024-03-16T10:05:29.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.201+0000 7f8121600640 1 -- 172.21.15.39:0/1625255691 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f811c09bae0 con 0x7f8124101380 2024-03-16T10:05:29.203 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:29.203 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T10:05:29.203 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2024-03-16T10:05:29.205 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:28 smithi039 systemd[1]: Starting Ceph mds.j for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:29.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 msgr2=0x7f810c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:29.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 0x7f810c0a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8118009c90 tx=0x7f8118005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 msgr2=0x7f81241a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 0x7f81241a12f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f811c009fd0 tx=0x7f811c00ef10 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 shutdown_connections 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f810c0a2800 0x7f810c0a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8124101380 0x7f81241a12f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81241009e0 0x7f812419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 --2- 172.21.15.39:0/1625255691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8124100080 0x7f8124199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.204+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 >> 172.21.15.39:0/1625255691 conn(0x7f81240fbc60 msgr2=0x7f81240fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.205+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 shutdown_connections 2024-03-16T10:05:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:29.205+0000 7f812bc00640 1 -- 172.21.15.39:0/1625255691 wait complete. 2024-03-16T10:05:29.504 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 podman[72228]: 2024-03-16 10:05:29.110476713 +0000 UTC m=+0.035147701 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:29.505 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 podman[72228]: 2024-03-16 10:05:29.231850881 +0000 UTC m=+0.156521901 container create e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 podman[72228]: 2024-03-16 10:05:29.502695969 +0000 UTC m=+0.427366996 container init e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 podman[72228]: 2024-03-16 10:05:29.514632922 +0000 UTC m=+0.439303935 container start e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mds[72248]: -- 172.21.15.39:0/3850064581 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641169cba40 con 0x56411699ec00 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: starting mds.j at 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 bash[72228]: e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a 2024-03-16T10:05:29.842 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:05:29 smithi039 systemd[1]: Started Ceph mds.j for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1625255691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:30.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:29 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1625255691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:30.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:29 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:30.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:30.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1625255691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:30.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:30.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:30.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:30.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:29 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:31.109 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:30 smithi039 ceph-mon[31510]: Deploying daemon mds.d on smithi039 2024-03-16T10:05:31.109 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:30 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6834/1951962450,v1:172.21.15.39:6835/1951962450] up:boot 2024-03-16T10:05:31.109 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:30 smithi039 ceph-mon[31510]: fsmap 3 up:standby 2024-03-16T10:05:31.109 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:30 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:05:31.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:30 smithi105 ceph-mon[36559]: Deploying daemon mds.d on smithi039 2024-03-16T10:05:31.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:30 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6834/1951962450,v1:172.21.15.39:6835/1951962450] up:boot 2024-03-16T10:05:31.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:30 smithi105 ceph-mon[36559]: fsmap 3 up:standby 2024-03-16T10:05:31.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:30 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:05:31.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:30 smithi194 ceph-mon[36445]: Deploying daemon mds.d on smithi039 2024-03-16T10:05:31.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:30 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6834/1951962450,v1:172.21.15.39:6835/1951962450] up:boot 2024-03-16T10:05:31.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:30 smithi194 ceph-mon[36445]: fsmap 3 up:standby 2024-03-16T10:05:31.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:30 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:05:31.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:31.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 -- 172.21.15.39:0/3783063309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8784101160 msgr2=0x7f87841015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 --2- 172.21.15.39:0/3783063309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8784101160 0x7f87841015e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f877c009ca0 tx=0x7f877c02c0f0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 -- 172.21.15.39:0/3783063309 shutdown_connections 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 --2- 172.21.15.39:0/3783063309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8784101160 0x7f87841015e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 --2- 172.21.15.39:0/3783063309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 0x7f8784100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 --2- 172.21.15.39:0/3783063309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87840ffe60 0x7f8784100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 -- 172.21.15.39:0/3783063309 >> 172.21.15.39:0/3783063309 conn(0x7f87840fba60 msgr2=0x7f87840fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 -- 172.21.15.39:0/3783063309 shutdown_connections 2024-03-16T10:05:31.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.449+0000 7f8783200640 1 -- 172.21.15.39:0/3783063309 wait complete. 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 Processor -- start 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 -- start start 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 0x7f8784199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8784101160 0x7f87841a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f878410f440 con 0x7f87840ffe60 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f878410f2c0 con 0x7f87841007c0 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.450+0000 7f8783200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f878410f5c0 con 0x7f8784101160 2024-03-16T10:05:31.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46742/0 (socket says 172.21.15.39:46742) 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 -- 172.21.15.39:0/2511471401 learned_addr learned my addr 172.21.15.39:0/2511471401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781400640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 0x7f8784199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8782800640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8784101160 0x7f87841a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 -- 172.21.15.39:0/2511471401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8784101160 msgr2=0x7f87841a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8784101160 0x7f87841a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 -- 172.21.15.39:0/2511471401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 msgr2=0x7f8784199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 0x7f8784199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 -- 172.21.15.39:0/2511471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87841a1670 con 0x7f87840ffe60 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.451+0000 7f8781e00640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f877400e9f0 tx=0x7f877400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.452+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400cd80 con 0x7f87840ffe60 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.452+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774010910 con 0x7f87840ffe60 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.452+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87841a1960 con 0x7f87840ffe60 2024-03-16T10:05:31.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.452+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774018990 con 0x7f87840ffe60 2024-03-16T10:05:31.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.452+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841a1f00 con 0x7f87840ffe60 2024-03-16T10:05:31.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.453+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878410ad00 con 0x7f87840ffe60 2024-03-16T10:05:31.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.455+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8774010ac0 con 0x7f87840ffe60 2024-03-16T10:05:31.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.457+0000 7f877aa00640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 0x7f87640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:31.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.458+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f87740ce890 con 0x7f87840ffe60 2024-03-16T10:05:31.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.458+0000 7f8781400640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 0x7f87640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:31.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.458+0000 7f8781400640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 0x7f87640a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f876c0095f0 tx=0x7f876c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:31.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.462+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f877409b760 con 0x7f87840ffe60 2024-03-16T10:05:31.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.711+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f878402d1b0 con 0x7f87840ffe60 2024-03-16T10:05:31.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.712+0000 7f877aa00640 1 -- 172.21.15.39:0/2511471401 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f878402d1b0 con 0x7f87840ffe60 2024-03-16T10:05:31.714 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:31.714 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-03-16T10:05:31.714 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2024-03-16T10:05:31.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 msgr2=0x7f87640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 0x7f87640a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f876c0095f0 tx=0x7f876c005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 msgr2=0x7f8784199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f877400e9f0 tx=0x7f877400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 shutdown_connections 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87640a2880 0x7f87640a4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8784101160 0x7f87841a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87841007c0 0x7f8784199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 --2- 172.21.15.39:0/2511471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87840ffe60 0x7f8784199a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 >> 172.21.15.39:0/2511471401 conn(0x7f87840fba60 msgr2=0x7f87840fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 shutdown_connections 2024-03-16T10:05:31.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:31.715+0000 7f8783200640 1 -- 172.21.15.39:0/2511471401 wait complete. 2024-03-16T10:05:31.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:31 smithi039 ceph-mon[31510]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:31.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:31 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2511471401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:32.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:31 smithi194 ceph-mon[36445]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:32.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:31 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2511471401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:32.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:31 smithi105 ceph-mon[36559]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:32.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:31 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2511471401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:32.709 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:32 smithi039 systemd[1]: Starting Ceph mds.d for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:33.378 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 podman[72579]: 2024-03-16 10:05:32.983137962 +0000 UTC m=+0.037221220 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:33.378 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 podman[72579]: 2024-03-16 10:05:33.114039793 +0000 UTC m=+0.168123092 container create f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:05:33.379 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 podman[72579]: 2024-03-16 10:05:33.377001123 +0000 UTC m=+0.431084458 container init f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:05:33.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:33.755 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 podman[72579]: 2024-03-16 10:05:33.387456537 +0000 UTC m=+0.441539838 container start f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:05:33.755 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mds[72598]: -- 172.21.15.39:0/3734944933 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642f21bda40 con 0x5642f2190c00 2024-03-16T10:05:33.755 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: starting mds.d at 2024-03-16T10:05:33.756 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 bash[72579]: f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a 2024-03-16T10:05:33.756 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:05:33 smithi039 systemd[1]: Started Ceph mds.d for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 -- 172.21.15.39:0/1047113146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff78107c20 msgr2=0x7eff78108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 --2- 172.21.15.39:0/1047113146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff78107c20 0x7eff78108040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff68009900 tx=0x7eff6802bc30 comp rx=0 tx=0).stop 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 -- 172.21.15.39:0/1047113146 shutdown_connections 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 --2- 172.21.15.39:0/1047113146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff781097d0 0x7eff78110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 --2- 172.21.15.39:0/1047113146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78108e10 0x7eff78109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 --2- 172.21.15.39:0/1047113146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff78107c20 0x7eff78108040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 -- 172.21.15.39:0/1047113146 >> 172.21.15.39:0/1047113146 conn(0x7eff78076bb0 msgr2=0x7eff78078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 -- 172.21.15.39:0/1047113146 shutdown_connections 2024-03-16T10:05:33.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.975+0000 7eff77e00640 1 -- 172.21.15.39:0/1047113146 wait complete. 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 Processor -- start 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 -- start start 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 0x7eff7819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff78108e10 0x7eff7819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 0x7eff781a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff781138f0 con 0x7eff78107c20 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff78113770 con 0x7eff781097d0 2024-03-16T10:05:33.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff77e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7eff78113a70 con 0x7eff78108e10 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.977+0000 7eff76a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 0x7eff7819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 0x7eff781a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff76a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 0x7eff7819df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46786/0 (socket says 172.21.15.39:46786) 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff76a00640 1 -- 172.21.15.39:0/649355132 learned_addr learned my addr 172.21.15.39:0/649355132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff76000640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff78108e10 0x7eff7819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 -- 172.21.15.39:0/649355132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff78108e10 msgr2=0x7eff7819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff78108e10 0x7eff7819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 -- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 msgr2=0x7eff7819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 0x7eff7819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:33.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 -- 172.21.15.39:0/649355132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff781a5b20 con 0x7eff781097d0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff77400640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 0x7eff781a54b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7eff70009fd0 tx=0x7eff7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7000ce20 con 0x7eff781097d0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.978+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff70005020 con 0x7eff781097d0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.979+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70010720 con 0x7eff781097d0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.979+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff781a5e10 con 0x7eff781097d0 2024-03-16T10:05:33.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.979+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff781a6320 con 0x7eff781097d0 2024-03-16T10:05:33.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.980+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7810eee0 con 0x7eff781097d0 2024-03-16T10:05:33.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.982+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7eff70005540 con 0x7eff781097d0 2024-03-16T10:05:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.983+0000 7eff6f400640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 0x7eff5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.983+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7eff700ceb40 con 0x7eff781097d0 2024-03-16T10:05:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.983+0000 7eff76a00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 0x7eff5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.984+0000 7eff76a00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 0x7eff5c0a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff68006010 tx=0x7eff6802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:33.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:33.987+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7eff7009c080 con 0x7eff781097d0 2024-03-16T10:05:34.166 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:34.166 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.166 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.166 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.166 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:34.167 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:34.167 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:33 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:34.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:33 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:34.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.232+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff7802d1b0 con 0x7eff781097d0 2024-03-16T10:05:34.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.233+0000 7eff6f400640 1 -- 172.21.15.39:0/649355132 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7eff7009b980 con 0x7eff781097d0 2024-03-16T10:05:34.235 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:34.235 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-03-16T10:05:34.235 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2024-03-16T10:05:34.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 msgr2=0x7eff5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 0x7eff5c0a4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7eff68006010 tx=0x7eff6802df90 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 msgr2=0x7eff781a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 0x7eff781a54b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7eff70009fd0 tx=0x7eff7000ef10 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 shutdown_connections 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7eff5c0a2880 0x7eff5c0a4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff781097d0 0x7eff781a54b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff78108e10 0x7eff7819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 --2- 172.21.15.39:0/649355132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff78107c20 0x7eff7819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 >> 172.21.15.39:0/649355132 conn(0x7eff78076bb0 msgr2=0x7eff7810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 shutdown_connections 2024-03-16T10:05:34.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:34.236+0000 7eff77e00640 1 -- 172.21.15.39:0/649355132 wait complete. 2024-03-16T10:05:34.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:34.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:33 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:35.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:34 smithi194 ceph-mon[36445]: Deploying daemon mds.e on smithi105 2024-03-16T10:05:35.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:34 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6836/2605661802,v1:172.21.15.39:6837/2605661802] up:boot 2024-03-16T10:05:35.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:34 smithi194 ceph-mon[36445]: fsmap 4 up:standby 2024-03-16T10:05:35.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:34 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:05:35.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:34 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/649355132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:34 smithi039 ceph-mon[31510]: Deploying daemon mds.e on smithi105 2024-03-16T10:05:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:34 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6836/2605661802,v1:172.21.15.39:6837/2605661802] up:boot 2024-03-16T10:05:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:34 smithi039 ceph-mon[31510]: fsmap 4 up:standby 2024-03-16T10:05:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:34 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:05:35.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:34 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/649355132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:35.330 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:34 smithi105 ceph-mon[36559]: Deploying daemon mds.e on smithi105 2024-03-16T10:05:35.331 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:34 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6836/2605661802,v1:172.21.15.39:6837/2605661802] up:boot 2024-03-16T10:05:35.331 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:34 smithi105 ceph-mon[36559]: fsmap 4 up:standby 2024-03-16T10:05:35.331 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:34 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:05:35.331 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:34 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/649355132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:36.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:35 smithi105 ceph-mon[36559]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:36.115 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 systemd[1]: Starting Ceph mds.e for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:36.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:35 smithi194 ceph-mon[36445]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:36.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:35 smithi039 ceph-mon[31510]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:36.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 -- 172.21.15.39:0/2021865346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc0773e0 msgr2=0x7f7fbc077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/2021865346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc077840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009900 tx=0x7f7fa402bc50 comp rx=0 tx=0).stop 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 -- 172.21.15.39:0/2021865346 shutdown_connections 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/2021865346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc077d80 0x7f7fbc078620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/2021865346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc077840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/2021865346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0761a0 0x7f7fbc0765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.490+0000 7f7fc1c00640 1 -- 172.21.15.39:0/2021865346 >> 172.21.15.39:0/2021865346 conn(0x7f7fbc0fef90 msgr2=0x7f7fbc1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:36.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.491+0000 7f7fc1c00640 1 -- 172.21.15.39:0/2021865346 shutdown_connections 2024-03-16T10:05:36.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.491+0000 7f7fc1c00640 1 -- 172.21.15.39:0/2021865346 wait complete. 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 Processor -- start 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 -- start start 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc0761a0 0x7f7fbc19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113ad0 con 0x7f7fbc0761a0 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113950 con 0x7f7fbc077d80 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fc1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113c50 con 0x7f7fbc0773e0 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fb2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fbaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc0761a0 0x7f7fbc19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:36.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.492+0000 7f7fb2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:45584/0 (socket says 172.21.15.39:45584) 2024-03-16T10:05:36.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fb2000640 1 -- 172.21.15.39:0/4149982883 learned_addr learned my addr 172.21.15.39:0/4149982883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:36.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:35902/0 (socket says 172.21.15.39:35902) 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 -- 172.21.15.39:0/4149982883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 msgr2=0x7f7fbc19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 -- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc0761a0 msgr2=0x7f7fbc19e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc0761a0 0x7f7fbc19e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 -- 172.21.15.39:0/4149982883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc1a5ce0 con 0x7f7fbc077d80 2024-03-16T10:05:36.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fbb400640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.493+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400ce20 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.494+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4005020 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.494+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4010720 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.494+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1a5fd0 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.494+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a6490 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.495+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc110370 con 0x7f7fbc077d80 2024-03-16T10:05:36.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.496+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7fb4005540 con 0x7f7fbc077d80 2024-03-16T10:05:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.498+0000 7f7fb3e00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 0x7f7fa00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.498+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7fb40cecd0 con 0x7f7fbc077d80 2024-03-16T10:05:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.498+0000 7f7fbaa00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 0x7f7fa00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:36.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.499+0000 7f7fbaa00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 0x7f7fa00a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009bd0 tx=0x7f7fac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:36.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.502+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7fb409c170 con 0x7f7fbc077d80 2024-03-16T10:05:36.509 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 podman[66575]: 2024-03-16 10:05:36.159266335 +0000 UTC m=+0.018085514 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:36.509 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 podman[66575]: 2024-03-16 10:05:36.286842916 +0000 UTC m=+0.145662106 container create 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-16T10:05:36.509 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 podman[66575]: 2024-03-16 10:05:36.506030925 +0000 UTC m=+0.364850089 container init 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T10:05:36.509 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 podman[66575]: 2024-03-16 10:05:36.509088832 +0000 UTC m=+0.367908002 container start 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307) 2024-03-16T10:05:36.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.750+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fbc02d1b0 con 0x7f7fbc077d80 2024-03-16T10:05:36.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.751+0000 7f7fb3e00640 1 -- 172.21.15.39:0/4149982883 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f7fb409ba70 con 0x7f7fbc077d80 2024-03-16T10:05:36.753 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:36.753 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-03-16T10:05:36.753 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 msgr2=0x7f7fa00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 0x7f7fa00a4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009bd0 tx=0x7f7fac005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 msgr2=0x7f7fbc1a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009fd0 tx=0x7f7fb400ef10 comp rx=0 tx=0).stop 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 shutdown_connections 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7fa00a2880 0x7f7fa00a4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fbc077d80 0x7f7fbc1a5670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7fbc0773e0 0x7f7fbc19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 --2- 172.21.15.39:0/4149982883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fbc0761a0 0x7f7fbc19e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:36.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 >> 172.21.15.39:0/4149982883 conn(0x7f7fbc0fef90 msgr2=0x7f7fbc100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:36.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 shutdown_connections 2024-03-16T10:05:36.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:36.754+0000 7f7fc1c00640 1 -- 172.21.15.39:0/4149982883 wait complete. 2024-03-16T10:05:36.864 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 ceph-mds[66594]: -- 172.21.15.105:0/3438074738 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db5ba5ba40 con 0x55db5ba2ec00 2024-03-16T10:05:36.865 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: starting mds.e at 2024-03-16T10:05:36.865 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 bash[66575]: 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 2024-03-16T10:05:36.865 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:05:36 smithi105 systemd[1]: Started Ceph mds.e for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:37.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: Deploying daemon mds.f on smithi194 2024-03-16T10:05:37.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4149982883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:37.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6834/106855660,v1:172.21.15.105:6835/106855660] up:boot 2024-03-16T10:05:37.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: fsmap 5 up:standby 2024-03-16T10:05:37.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:37 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:05:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: Deploying daemon mds.f on smithi194 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4149982883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6834/106855660,v1:172.21.15.105:6835/106855660] up:boot 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: fsmap 5 up:standby 2024-03-16T10:05:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:37 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: Deploying daemon mds.f on smithi194 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4149982883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6834/106855660,v1:172.21.15.105:6835/106855660] up:boot 2024-03-16T10:05:38.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: fsmap 5 up:standby 2024-03-16T10:05:38.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:37 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:05:38.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 -- 172.21.15.39:0/3824068048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 msgr2=0x7f758c077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 --2- 172.21.15.39:0/3824068048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c077840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7578009860 tx=0x7f757802bb70 comp rx=0 tx=0).stop 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 -- 172.21.15.39:0/3824068048 shutdown_connections 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 --2- 172.21.15.39:0/3824068048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c077d80 0x7f758c078620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 --2- 172.21.15.39:0/3824068048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c077840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 --2- 172.21.15.39:0/3824068048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c0761a0 0x7f758c0765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 -- 172.21.15.39:0/3824068048 >> 172.21.15.39:0/3824068048 conn(0x7f758c0fef90 msgr2=0x7f758c1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 -- 172.21.15.39:0/3824068048 shutdown_connections 2024-03-16T10:05:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.010+0000 7f7592a00640 1 -- 172.21.15.39:0/3824068048 wait complete. 2024-03-16T10:05:39.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.011+0000 7f7592a00640 1 Processor -- start 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.011+0000 7f7592a00640 1 -- start start 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c077d80 0x7f758c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f758c113a50 con 0x7f758c077d80 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1138d0 con 0x7f758c0773e0 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7592a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f758c113bd0 con 0x7f758c0761a0 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:45614/0 (socket says 172.21.15.39:45614) 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 -- 172.21.15.39:0/1989208267 learned_addr learned my addr 172.21.15.39:0/1989208267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f7590a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c077d80 0x7f758c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:39.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758b400640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 -- 172.21.15.39:0/1989208267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 msgr2=0x7f758c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 -- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c077d80 msgr2=0x7f758c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c077d80 0x7f758c1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.012+0000 7f758be00640 1 -- 172.21.15.39:0/1989208267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c1a5d30 con 0x7f758c0761a0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f758be00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cd80 con 0x7f758c0761a0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a5fc0 con 0x7f758c0761a0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757c010910 con 0x7f758c0761a0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0188f0 con 0x7f758c0761a0 2024-03-16T10:05:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.013+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c1a64d0 con 0x7f758c0761a0 2024-03-16T10:05:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.014+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c110370 con 0x7f758c0761a0 2024-03-16T10:05:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.016+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f757c010ac0 con 0x7f758c0761a0 2024-03-16T10:05:39.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.017+0000 7f7588c00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 0x7f75700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.017+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f757c0ce990 con 0x7f758c0761a0 2024-03-16T10:05:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.017+0000 7f758b400640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 0x7f75700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.018+0000 7f758b400640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 0x7f75700a4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7578009830 tx=0x7f7578034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:39.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.021+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f757c09b7e0 con 0x7f758c0761a0 2024-03-16T10:05:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.267+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f758c02d1b0 con 0x7f758c0761a0 2024-03-16T10:05:39.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.268+0000 7f7588c00640 1 -- 172.21.15.39:0/1989208267 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f758c02d1b0 con 0x7f758c0761a0 2024-03-16T10:05:39.270 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:39.270 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-16T10:05:39.270 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.270+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 msgr2=0x7f75700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.270+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 0x7f75700a4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7578009830 tx=0x7f7578034ab0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.270+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 msgr2=0x7f758c19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.270+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 shutdown_connections 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f75700a2880 0x7f75700a4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c077d80 0x7f758c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f758c0773e0 0x7f758c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 --2- 172.21.15.39:0/1989208267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f758c0761a0 0x7f758c19e110 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:39.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 >> 172.21.15.39:0/1989208267 conn(0x7f758c0fef90 msgr2=0x7f758c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:39.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 shutdown_connections 2024-03-16T10:05:39.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:39.271+0000 7f7592a00640 1 -- 172.21.15.39:0/1989208267 wait complete. 2024-03-16T10:05:39.682 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:05:39 smithi194 systemd[1]: Started Ceph mds.f for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:39 smithi194 ceph-mon[36445]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:39 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1989208267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:39 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:39 smithi039 ceph-mon[31510]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:39 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1989208267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:39 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:40.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:39 smithi105 ceph-mon[36559]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:39 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1989208267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:40.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:39 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6834/3003167044,v1:172.21.15.194:6835/3003167044] up:boot 2024-03-16T10:05:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: fsmap 6 up:standby 2024-03-16T10:05:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:05:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:40.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:40.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:40.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:40.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:40 smithi194 ceph-mon[36445]: Deploying daemon mds.c on smithi194 2024-03-16T10:05:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6834/3003167044,v1:172.21.15.194:6835/3003167044] up:boot 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: fsmap 6 up:standby 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:40 smithi039 ceph-mon[31510]: Deploying daemon mds.c on smithi194 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6834/3003167044,v1:172.21.15.194:6835/3003167044] up:boot 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: fsmap 6 up:standby 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:41.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:40 smithi105 ceph-mon[36559]: Deploying daemon mds.c on smithi194 2024-03-16T10:05:41.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 -- 172.21.15.39:0/341752599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01049f0 msgr2=0x7f49a0104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 --2- 172.21.15.39:0/341752599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01049f0 0x7f49a0104e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4988009900 tx=0x7f498802bc30 comp rx=0 tx=0).stop 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 -- 172.21.15.39:0/341752599 shutdown_connections 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 --2- 172.21.15.39:0/341752599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01053b0 0x7f49a010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 --2- 172.21.15.39:0/341752599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01049f0 0x7f49a0104e70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 --2- 172.21.15.39:0/341752599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 -- 172.21.15.39:0/341752599 >> 172.21.15.39:0/341752599 conn(0x7f49a00fed70 msgr2=0x7f49a01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 -- 172.21.15.39:0/341752599 shutdown_connections 2024-03-16T10:05:41.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.528+0000 7f499fe00640 1 -- 172.21.15.39:0/341752599 wait complete. 2024-03-16T10:05:41.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.529+0000 7f499fe00640 1 Processor -- start 2024-03-16T10:05:41.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.529+0000 7f499fe00640 1 -- start start 2024-03-16T10:05:41.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:41.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 0x7f49a019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 0x7f49a01a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49a010f4b0 con 0x7f49a01053b0 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49a010f330 con 0x7f49a0103800 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49a010f630 con 0x7f49a01049f0 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 0x7f49a01a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f4996000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 0x7f49a019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39392/0 (socket says 172.21.15.39:39392) 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 -- 172.21.15.39:0/2362039783 learned_addr learned my addr 172.21.15.39:0/2362039783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:41.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 -- 172.21.15.39:0/2362039783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 msgr2=0x7f49a019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 0x7f49a019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 -- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 msgr2=0x7f49a01a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 0x7f49a01a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499ea00640 1 -- 172.21.15.39:0/2362039783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a01a1740 con 0x7f49a0103800 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f4996000640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 0x7f49a019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.530+0000 7f499f400640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 0x7f49a01a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f499ea00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f499000ebc0 tx=0x7f499000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990003b70 con 0x7f49a0103800 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a01a1a30 con 0x7f49a0103800 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4990003d10 con 0x7f49a0103800 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990019980 con 0x7f49a0103800 2024-03-16T10:05:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.531+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a01a1f50 con 0x7f49a0103800 2024-03-16T10:05:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.532+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a0068940 con 0x7f49a0103800 2024-03-16T10:05:41.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.534+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4990004b60 con 0x7f49a0103800 2024-03-16T10:05:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.536+0000 7f4997e00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 0x7f49840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.536+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f49900cee20 con 0x7f49a0103800 2024-03-16T10:05:41.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.536+0000 7f4996000640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 0x7f49840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.537+0000 7f4996000640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 0x7f49840a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f498802c240 tx=0x7f498802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.538+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f499009c2c0 con 0x7f49a0103800 2024-03-16T10:05:41.772 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:41 smithi194 ceph-mon[36445]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:41.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.786+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49a002d1b0 con 0x7f49a0103800 2024-03-16T10:05:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.787+0000 7f4997e00640 1 -- 172.21.15.39:0/2362039783 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f49a002d1b0 con 0x7f49a0103800 2024-03-16T10:05:41.788 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:41.789 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-16T10:05:41.789 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 7 2024-03-16T10:05:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 msgr2=0x7f49840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 0x7f49840a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f498802c240 tx=0x7f498802df90 comp rx=0 tx=0).stop 2024-03-16T10:05:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 msgr2=0x7f49a0199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f499000ebc0 tx=0x7f499000c6a0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 shutdown_connections 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49840a2800 0x7f49840a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a01053b0 0x7f49a01a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49a01049f0 0x7f49a019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 --2- 172.21.15.39:0/2362039783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49a0103800 0x7f49a0199b20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 >> 172.21.15.39:0/2362039783 conn(0x7f49a00fed70 msgr2=0x7f49a0107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 shutdown_connections 2024-03-16T10:05:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:41.790+0000 7f499fe00640 1 -- 172.21.15.39:0/2362039783 wait complete. 2024-03-16T10:05:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:41 smithi039 ceph-mon[31510]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:42.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:41 smithi105 ceph-mon[36559]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:42.168 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:41 smithi194 systemd[1]: Starting Ceph mds.c for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:42.419 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 podman[67661]: 2024-03-16 10:05:42.070256264 +0000 UTC m=+0.017515142 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:42.419 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 podman[67661]: 2024-03-16 10:05:42.188626 +0000 UTC m=+0.135884883 container create a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-16T10:05:42.419 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 podman[67661]: 2024-03-16 10:05:42.416286592 +0000 UTC m=+0.363545461 container init a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T10:05:42.419 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 podman[67661]: 2024-03-16 10:05:42.419067404 +0000 UTC m=+0.366326263 container start a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T10:05:42.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mds[67680]: -- 172.21.15.194:0/3432178382 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5aab07a40 con 0x55d5aaadb000 2024-03-16T10:05:42.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: starting mds.c at 2024-03-16T10:05:42.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 bash[67661]: a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb 2024-03-16T10:05:42.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:05:42 smithi194 systemd[1]: Started Ceph mds.c for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2362039783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:43.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:43.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:42 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:43.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2362039783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:43.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:43.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:43.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:42 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2362039783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:43.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:43.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:42 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:43.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:43 smithi105 ceph-mon[36559]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:43 smithi105 ceph-mon[36559]: Deploying daemon mds.h on smithi105 2024-03-16T10:05:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:43 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:boot 2024-03-16T10:05:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:43 smithi105 ceph-mon[36559]: fsmap 7 up:standby 2024-03-16T10:05:43.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:43 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:05:44.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:43 smithi039 ceph-mon[31510]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:43 smithi039 ceph-mon[31510]: Deploying daemon mds.h on smithi105 2024-03-16T10:05:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:43 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:boot 2024-03-16T10:05:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:43 smithi039 ceph-mon[31510]: fsmap 7 up:standby 2024-03-16T10:05:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:43 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 -- 172.21.15.39:0/3812851387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78104cd0 msgr2=0x7f7f781070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3812851387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78104cd0 0x7f7f781070c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009860 tx=0x7f7f6402bb70 comp rx=0 tx=0).stop 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 -- 172.21.15.39:0/3812851387 shutdown_connections 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3812851387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78107600 0x7f7f781099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3812851387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78104cd0 0x7f7f781070c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3812851387 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 0x7f7f78104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 -- 172.21.15.39:0/3812851387 >> 172.21.15.39:0/3812851387 conn(0x7f7f780fbd10 msgr2=0x7f7f780fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.045+0000 7f7f7e400640 1 -- 172.21.15.39:0/3812851387 shutdown_connections 2024-03-16T10:05:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.046+0000 7f7f7e400640 1 -- 172.21.15.39:0/3812851387 wait complete. 2024-03-16T10:05:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.046+0000 7f7f7e400640 1 Processor -- start 2024-03-16T10:05:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 -- start start 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 0x7f7f78199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 0x7f7f781a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7810d4e0 con 0x7f7f78104cd0 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7810d360 con 0x7f7f78107600 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f7e400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7810d660 con 0x7f7f781023a0 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f76a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f76a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41522/0 (socket says 172.21.15.39:41522) 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f76a00640 1 -- 172.21.15.39:0/3197814379 learned_addr learned my addr 172.21.15.39:0/3197814379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f77400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 0x7f7f78199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.047+0000 7f7f77e00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 0x7f7f781a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:44.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 -- 172.21.15.39:0/3197814379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 msgr2=0x7f7f78199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:44.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 0x7f7f78199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 -- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 msgr2=0x7f7f7819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 -- 172.21.15.39:0/3197814379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f781a1990 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f76a00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f77e00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 0x7f7f781a1320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fd0 tx=0x7f7f7000eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70004220 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7000cd90 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70004a00 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f781a1c80 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.048+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f781a21d0 con 0x7f7f78107600 2024-03-16T10:05:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.050+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f781090f0 con 0x7f7f78107600 2024-03-16T10:05:44.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.051+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7f70004ba0 con 0x7f7f78107600 2024-03-16T10:05:44.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.052+0000 7f7f6fe00640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 0x7f7f5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:44.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.053+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7f700ce3c0 con 0x7f7f78107600 2024-03-16T10:05:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.053+0000 7f7f77400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 0x7f7f5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.053+0000 7f7f77400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 0x7f7f5c0a4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f68005fd0 tx=0x7f7f68009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:44.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.057+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f7009b860 con 0x7f7f78107600 2024-03-16T10:05:44.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:43 smithi194 ceph-mon[36445]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:44.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:43 smithi194 ceph-mon[36445]: Deploying daemon mds.h on smithi105 2024-03-16T10:05:44.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:43 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:boot 2024-03-16T10:05:44.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:43 smithi194 ceph-mon[36445]: fsmap 7 up:standby 2024-03-16T10:05:44.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:43 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:05:44.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.301+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f780623e0 con 0x7f7f78107600 2024-03-16T10:05:44.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.302+0000 7f7f6fe00640 1 -- 172.21.15.39:0/3197814379 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f7f7009b160 con 0x7f7f78107600 2024-03-16T10:05:44.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:44.304 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-16T10:05:44.304 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.304+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 msgr2=0x7f7f5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.304+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 0x7f7f5c0a4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f68005fd0 tx=0x7f7f68009500 comp rx=0 tx=0).stop 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.304+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 msgr2=0x7f7f781a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.304+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 0x7f7f781a1320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009fd0 tx=0x7f7f7000eba0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 shutdown_connections 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7f5c0a2880 0x7f7f5c0a4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f78107600 0x7f7f781a1320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f78104cd0 0x7f7f7819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 --2- 172.21.15.39:0/3197814379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f781023a0 0x7f7f78199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:44.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 >> 172.21.15.39:0/3197814379 conn(0x7f7f780fbd10 msgr2=0x7f7f78101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:44.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 shutdown_connections 2024-03-16T10:05:44.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:44.305+0000 7f7f7e400640 1 -- 172.21.15.39:0/3197814379 wait complete. 2024-03-16T10:05:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:44 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3197814379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:44 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3197814379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:45.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:44 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3197814379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:45.615 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:05:45 smithi105 systemd[1]: Started Ceph mds.h for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:46.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:46.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:45 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:46.026 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:46.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:45 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:46.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:46.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:45 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:46.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:46.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 -- 172.21.15.39:0/3403312804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 msgr2=0x7f0888110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:46.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 --2- 172.21.15.39:0/3403312804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f0888110060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0880009ca0 tx=0x7f088002c100 comp rx=0 tx=0).stop 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 -- 172.21.15.39:0/3403312804 shutdown_connections 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 --2- 172.21.15.39:0/3403312804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f0888110060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 --2- 172.21.15.39:0/3403312804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f0888109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 --2- 172.21.15.39:0/3403312804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f0888108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 -- 172.21.15.39:0/3403312804 >> 172.21.15.39:0/3403312804 conn(0x7f0888076bb0 msgr2=0x7f0888078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 -- 172.21.15.39:0/3403312804 shutdown_connections 2024-03-16T10:05:46.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.556+0000 7f088cc00640 1 -- 172.21.15.39:0/3403312804 wait complete. 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 Processor -- start 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 -- start start 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f088819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f088819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f08881a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08881138a0 con 0x7f08881097d0 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0888113720 con 0x7f0888107c20 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.557+0000 7f088cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0888113a20 con 0x7f0888108e10 2024-03-16T10:05:46.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f0887400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f088819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f0887400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f088819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39440/0 (socket says 172.21.15.39:39440) 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f0887400640 1 -- 172.21.15.39:0/3666220200 learned_addr learned my addr 172.21.15.39:0/3666220200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f088819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f0887e00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f08881a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 -- 172.21.15.39:0/3666220200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 msgr2=0x7f088819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f088819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 -- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 msgr2=0x7f08881a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f08881a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 -- 172.21.15.39:0/3666220200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08881a5b70 con 0x7f0888108e10 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f0887e00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f08881a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.558+0000 7f087ea00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f088819e490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f087000e9f0 tx=0x7f087000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.559+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000ccc0 con 0x7f0888108e10 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.559+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870005020 con 0x7f0888108e10 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.559+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08881a5e60 con 0x7f0888108e10 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.559+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08700049e0 con 0x7f0888108e10 2024-03-16T10:05:46.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.559+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a62d0 con 0x7f0888108e10 2024-03-16T10:05:46.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.560+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088810eee0 con 0x7f0888108e10 2024-03-16T10:05:46.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.562+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0870005560 con 0x7f0888108e10 2024-03-16T10:05:46.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.563+0000 7f0884c00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 0x7f08680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:46.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.563+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f08700ce5a0 con 0x7f0888108e10 2024-03-16T10:05:46.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.563+0000 7f0887400640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 0x7f08680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:46.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.564+0000 7f0887400640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 0x7f08680a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08780058c0 tx=0x7f0878005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:46.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.567+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f08700051d0 con 0x7f0888108e10 2024-03-16T10:05:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.820+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0888062480 con 0x7f0888108e10 2024-03-16T10:05:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.821+0000 7f0884c00640 1 -- 172.21.15.39:0/3666220200 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f087009ba40 con 0x7f0888108e10 2024-03-16T10:05:46.822 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:46.823 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-16T10:05:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2024-03-16T10:05:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 msgr2=0x7f08680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 0x7f08680a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08780058c0 tx=0x7f0878005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 msgr2=0x7f088819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f088819e490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f087000e9f0 tx=0x7f087000eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 shutdown_connections 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08680a2800 0x7f08680a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08881097d0 0x7f08881a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888108e10 0x7f088819e490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 --2- 172.21.15.39:0/3666220200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0888107c20 0x7f088819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 >> 172.21.15.39:0/3666220200 conn(0x7f0888076bb0 msgr2=0x7f088810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 shutdown_connections 2024-03-16T10:05:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:46.824+0000 7f088cc00640 1 -- 172.21.15.39:0/3666220200 wait complete. 2024-03-16T10:05:47.040 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:46 smithi194 ceph-mon[36445]: Deploying daemon mds.i on smithi194 2024-03-16T10:05:47.041 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:46 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:boot 2024-03-16T10:05:47.041 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:46 smithi194 ceph-mon[36445]: fsmap 8 up:standby 2024-03-16T10:05:47.041 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:46 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:05:47.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:46 smithi105 ceph-mon[36559]: Deploying daemon mds.i on smithi194 2024-03-16T10:05:47.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:46 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:boot 2024-03-16T10:05:47.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:46 smithi105 ceph-mon[36559]: fsmap 8 up:standby 2024-03-16T10:05:47.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:46 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:05:47.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:46 smithi039 ceph-mon[31510]: Deploying daemon mds.i on smithi194 2024-03-16T10:05:47.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:46 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:boot 2024-03-16T10:05:47.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:46 smithi039 ceph-mon[31510]: fsmap 8 up:standby 2024-03-16T10:05:47.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:46 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:05:47.706 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:47 smithi194 systemd[1]: Starting Ceph mds.i for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:48.062 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:47 smithi194 ceph-mon[36445]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:48.062 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:47 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3666220200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:48.063 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:47 smithi194 podman[67968]: 2024-03-16 10:05:47.719162286 +0000 UTC m=+0.017627935 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:48.063 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:47 smithi194 podman[67968]: 2024-03-16 10:05:47.826640165 +0000 UTC m=+0.125105791 container create 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T10:05:48.063 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 podman[67968]: 2024-03-16 10:05:48.062607904 +0000 UTC m=+0.361073557 container init 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-16T10:05:48.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:47 smithi105 ceph-mon[36559]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:47 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3666220200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:48.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:47 smithi039 ceph-mon[31510]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:48.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:47 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3666220200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:48.456 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 podman[67968]: 2024-03-16 10:05:48.065333694 +0000 UTC m=+0.363799321 container start 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T10:05:48.456 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 ceph-mds[67987]: -- 172.21.15.194:0/3188826779 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639e08e9a40 con 0x5639e08bcc00 2024-03-16T10:05:48.456 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: starting mds.i at 2024-03-16T10:05:48.456 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 bash[67968]: 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 2024-03-16T10:05:48.456 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:05:48 smithi194 systemd[1]: Started Ceph mds.i for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:48.845 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 -- 172.21.15.39:0/51231455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 msgr2=0x7fab34074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 --2- 172.21.15.39:0/51231455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34074620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab20009860 tx=0x7fab2002bc10 comp rx=0 tx=0).stop 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 -- 172.21.15.39:0/51231455 shutdown_connections 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 --2- 172.21.15.39:0/51231455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab34109690 0x7fab3410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 --2- 172.21.15.39:0/51231455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34074620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 --2- 172.21.15.39:0/51231455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab340757a0 0x7fab34073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 -- 172.21.15.39:0/51231455 >> 172.21.15.39:0/51231455 conn(0x7fab340fcd50 msgr2=0x7fab340ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:49.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 -- 172.21.15.39:0/51231455 shutdown_connections 2024-03-16T10:05:49.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.062+0000 7fab3b000640 1 -- 172.21.15.39:0/51231455 wait complete. 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.063+0000 7fab3b000640 1 Processor -- start 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 -- start start 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 0x7fab3419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab34109690 0x7fab341a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410f640 con 0x7fab340757a0 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410f4c0 con 0x7fab340741c0 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab3b000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410f7c0 con 0x7fab34109690 2024-03-16T10:05:49.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab33e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 0x7fab3419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab33e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39450/0 (socket says 172.21.15.39:39450) 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab33e00640 1 -- 172.21.15.39:0/2756872371 learned_addr learned my addr 172.21.15.39:0/2756872371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.064+0000 7fab39000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab34109690 0x7fab341a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 -- 172.21.15.39:0/2756872371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab34109690 msgr2=0x7fab341a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab34109690 0x7fab341a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 -- 172.21.15.39:0/2756872371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 msgr2=0x7fab34199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 -- 172.21.15.39:0/2756872371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341a1900 con 0x7fab340757a0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33e00640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.065+0000 7fab33400640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 0x7fab3419a230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab2002c150 tx=0x7fab200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.066+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20041020 con 0x7fab340757a0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.066+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341a1b90 con 0x7fab340757a0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.066+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab200035f0 con 0x7fab340757a0 2024-03-16T10:05:49.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.066+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20039040 con 0x7fab340757a0 2024-03-16T10:05:49.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.066+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a2080 con 0x7fab340757a0 2024-03-16T10:05:49.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.067+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab34108450 con 0x7fab340757a0 2024-03-16T10:05:49.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.069+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fab20044020 con 0x7fab340757a0 2024-03-16T10:05:49.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.070+0000 7fab30c00640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 0x7fab180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:49.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.070+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fab200efdc0 con 0x7fab340757a0 2024-03-16T10:05:49.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.070+0000 7fab33e00640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 0x7fab180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:49.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.071+0000 7fab33e00640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 0x7fab180a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab3419b1c0 tx=0x7fab24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:49.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.079+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fab2004b020 con 0x7fab340757a0 2024-03-16T10:05:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.333+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab3402d1b0 con 0x7fab340757a0 2024-03-16T10:05:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.334+0000 7fab30c00640 1 -- 172.21.15.39:0/2756872371 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fab200bd260 con 0x7fab340757a0 2024-03-16T10:05:49.336 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:49.336 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T10:05:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-03-16T10:05:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 msgr2=0x7fab180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 0x7fab180a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fab3419b1c0 tx=0x7fab24005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 msgr2=0x7fab3419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 0x7fab3419a230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab2002c150 tx=0x7fab200042f0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 shutdown_connections 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fab180a2800 0x7fab180a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab34109690 0x7fab341a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab340757a0 0x7fab3419a230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 --2- 172.21.15.39:0/2756872371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab340741c0 0x7fab34199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 >> 172.21.15.39:0/2756872371 conn(0x7fab340fcd50 msgr2=0x7fab34105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 shutdown_connections 2024-03-16T10:05:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:49.337+0000 7fab3b000640 1 -- 172.21.15.39:0/2756872371 wait complete. 2024-03-16T10:05:49.494 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: Deploying daemon mds.g on smithi039 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:boot 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: fsmap 9 up:standby 2024-03-16T10:05:49.495 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:49 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:05:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: Deploying daemon mds.g on smithi039 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:boot 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: fsmap 9 up:standby 2024-03-16T10:05:49.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:49 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:49.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: Deploying daemon mds.g on smithi039 2024-03-16T10:05:49.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:boot 2024-03-16T10:05:49.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: fsmap 9 up:standby 2024-03-16T10:05:49.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:49 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:05:50.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:50 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2756872371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:50.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:50 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2756872371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:50.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:50 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2756872371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:51.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:51.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:51 smithi039 ceph-mon[31510]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 -- 172.21.15.39:0/2613967931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 msgr2=0x7f8164106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 --2- 172.21.15.39:0/2613967931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164106f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8150009900 tx=0x7f815002bc30 comp rx=0 tx=0).stop 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 -- 172.21.15.39:0/2613967931 shutdown_connections 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 --2- 172.21.15.39:0/2613967931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81641074b0 0x7f8164109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 --2- 172.21.15.39:0/2613967931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164106f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 --2- 172.21.15.39:0/2613967931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8164102250 0x7f8164104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 -- 172.21.15.39:0/2613967931 >> 172.21.15.39:0/2613967931 conn(0x7f81640fbc40 msgr2=0x7f81640fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:51.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 -- 172.21.15.39:0/2613967931 shutdown_connections 2024-03-16T10:05:51.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.564+0000 7f816a400640 1 -- 172.21.15.39:0/2613967931 wait complete. 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.565+0000 7f816a400640 1 Processor -- start 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.565+0000 7f816a400640 1 -- start start 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 0x7f8164197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81641074b0 0x7f816419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f816410d450 con 0x7f8164104b80 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f816410d2d0 con 0x7f81641074b0 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f816a400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f816410d5d0 con 0x7f8164102250 2024-03-16T10:05:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8162a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 0x7f8164197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8162a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43200/0 (socket says 172.21.15.39:43200) 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8162a00640 1 -- 172.21.15.39:0/3617590953 learned_addr learned my addr 172.21.15.39:0/3617590953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163e00640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81641074b0 0x7f816419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 -- 172.21.15.39:0/3617590953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81641074b0 msgr2=0x7f816419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81641074b0 0x7f816419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 -- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 msgr2=0x7f8164198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8163400640 1 -- 172.21.15.39:0/3617590953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816419f710 con 0x7f8164102250 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.566+0000 7f8162a00640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f8163400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 0x7f8164197af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8154009bc0 tx=0x7f815400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815401e070 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81540198f0 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641000a0 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540187e0 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.567+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641005c0 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.568+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81640fd730 con 0x7f8164102250 2024-03-16T10:05:51.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.570+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8154021070 con 0x7f8164102250 2024-03-16T10:05:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.571+0000 7f815be00640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 0x7f81480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.572+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81540d1d80 con 0x7f8164102250 2024-03-16T10:05:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.572+0000 7f8162a00640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 0x7f81480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.572+0000 7f8162a00640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 0x7f81480a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f815002c8e0 tx=0x7f8150005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:51.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.576+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f815409ec70 con 0x7f8164102250 2024-03-16T10:05:51.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:51 smithi105 ceph-mon[36559]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:51 smithi194 ceph-mon[36445]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.824+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f816402d1b0 con 0x7f8164102250 2024-03-16T10:05:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.825+0000 7f815be00640 1 -- 172.21.15.39:0/3617590953 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f81540a2d10 con 0x7f8164102250 2024-03-16T10:05:51.827 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:51.827 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T10:05:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 msgr2=0x7f81480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 0x7f81480a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f815002c8e0 tx=0x7f8150005b50 comp rx=0 tx=0).stop 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 msgr2=0x7f8164197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 0x7f8164197af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8154009bc0 tx=0x7f815400b690 comp rx=0 tx=0).stop 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 shutdown_connections 2024-03-16T10:05:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f81480a2800 0x7f81480a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81641074b0 0x7f816419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8164104b80 0x7f8164198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 --2- 172.21.15.39:0/3617590953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8164102250 0x7f8164197af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.828+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 >> 172.21.15.39:0/3617590953 conn(0x7f81640fbc40 msgr2=0x7f8164102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.829+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 shutdown_connections 2024-03-16T10:05:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:51.829+0000 7f816a400640 1 -- 172.21.15.39:0/3617590953 wait complete. 2024-03-16T10:05:52.255 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:05:52 smithi039 systemd[1]: Started Ceph mds.g for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:52.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3617590953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:52.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3617590953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:52.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3617590953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6838/428058525,v1:172.21.15.39:6839/428058525] up:boot 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: fsmap 10 up:standby 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.546 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:53.546 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:53.546 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:53.546 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:53 smithi039 ceph-mon[31510]: Deploying daemon mds.a on smithi039 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6838/428058525,v1:172.21.15.39:6839/428058525] up:boot 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: fsmap 10 up:standby 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:53.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:53 smithi105 ceph-mon[36559]: Deploying daemon mds.a on smithi039 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6838/428058525,v1:172.21.15.39:6839/428058525] up:boot 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: fsmap 10 up:standby 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:53.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:53.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:53 smithi194 ceph-mon[36445]: Deploying daemon mds.a on smithi039 2024-03-16T10:05:53.850 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:54.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 -- 172.21.15.39:0/947538041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2000103190 msgr2=0x7f2000109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 --2- 172.21.15.39:0/947538041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2000103190 0x7f2000109a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009ca0 tx=0x7f1ff802c0f0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 -- 172.21.15.39:0/947538041 shutdown_connections 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 --2- 172.21.15.39:0/947538041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2000103190 0x7f2000109a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 --2- 172.21.15.39:0/947538041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20001027d0 0x7f2000102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 --2- 172.21.15.39:0/947538041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001015e0 0x7f2000101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 -- 172.21.15.39:0/947538041 >> 172.21.15.39:0/947538041 conn(0x7f20000fcb70 msgr2=0x7f20000fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 -- 172.21.15.39:0/947538041 shutdown_connections 2024-03-16T10:05:54.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.085+0000 7f2006600640 1 -- 172.21.15.39:0/947538041 wait complete. 2024-03-16T10:05:54.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.086+0000 7f2006600640 1 Processor -- start 2024-03-16T10:05:54.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.086+0000 7f2006600640 1 -- start start 2024-03-16T10:05:54.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.086+0000 7f2006600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:54.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.086+0000 7f2006600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001027d0 0x7f2000199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2006600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 0x7f20001a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2006600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f200010d240 con 0x7f20001015e0 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2006600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f200010d0c0 con 0x7f2000103190 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2006600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f200010d3c0 con 0x7f20001027d0 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43240/0 (socket says 172.21.15.39:43240) 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 -- 172.21.15.39:0/4163941630 learned_addr learned my addr 172.21.15.39:0/4163941630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f1fffe00640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001027d0 0x7f2000199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005c00640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 0x7f20001a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 -- 172.21.15.39:0/4163941630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001027d0 msgr2=0x7f2000199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001027d0 0x7f2000199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 -- 172.21.15.39:0/4163941630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 msgr2=0x7f20001a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:54.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 0x7f20001a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005200640 1 -- 172.21.15.39:0/4163941630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20001a16a0 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.087+0000 7f2005c00640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 0x7f20001a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f2005200640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400d770 tx=0x7f1ff400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4017070 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff4009c30 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001a1990 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4003dd0 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.088+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a1f30 con 0x7f20001015e0 2024-03-16T10:05:54.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.089+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20001a2220 con 0x7f20001015e0 2024-03-16T10:05:54.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.091+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1ff4003f70 con 0x7f20001015e0 2024-03-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.093+0000 7f1ffd600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 0x7f1fe80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.093+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1ff40ce320 con 0x7f20001015e0 2024-03-16T10:05:54.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.093+0000 7f1fffe00640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 0x7f1fe80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:54.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.094+0000 7f1fffe00640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 0x7f1fe80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f200019af90 tx=0x7f1fec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:54.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.096+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ff4009de0 con 0x7f20001015e0 2024-03-16T10:05:54.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.360+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20001a23f0 con 0x7f20001015e0 2024-03-16T10:05:54.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.362+0000 7f1ffd600640 1 -- 172.21.15.39:0/4163941630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f1ff409b1f0 con 0x7f20001015e0 2024-03-16T10:05:54.364 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:54.364 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T10:05:54.364 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 11 2024-03-16T10:05:54.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.364+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 msgr2=0x7f1fe80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.364+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 0x7f1fe80a4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f200019af90 tx=0x7f1fec005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.364+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 msgr2=0x7f2000199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.364+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400d770 tx=0x7f1ff400dc30 comp rx=0 tx=0).stop 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 shutdown_connections 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1fe80a2880 0x7f1fe80a4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2000103190 0x7f20001a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20001027d0 0x7f2000199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 --2- 172.21.15.39:0/4163941630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001015e0 0x7f2000199a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:54.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 >> 172.21.15.39:0/4163941630 conn(0x7f20000fcb70 msgr2=0x7f2000073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:54.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 shutdown_connections 2024-03-16T10:05:54.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:54.365+0000 7f2006600640 1 -- 172.21.15.39:0/4163941630 wait complete. 2024-03-16T10:05:55.481 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:55 smithi039 ceph-mon[31510]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:55.481 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4163941630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:55.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:55 smithi105 ceph-mon[36559]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:55.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4163941630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:55 smithi194 ceph-mon[36445]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:55.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4163941630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:55.755 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:55 smithi039 systemd[1]: Starting Ceph mds.a for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:56.151 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:55 smithi039 podman[73563]: 2024-03-16 10:05:55.761821984 +0000 UTC m=+0.041568998 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:56.151 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:55 smithi039 podman[73563]: 2024-03-16 10:05:55.878946744 +0000 UTC m=+0.158693759 container create fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T10:05:56.151 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 podman[73563]: 2024-03-16 10:05:56.149726836 +0000 UTC m=+0.429473891 container init fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T10:05:56.385 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:56.475 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 podman[73563]: 2024-03-16 10:05:56.159794032 +0000 UTC m=+0.439541017 container start fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-16T10:05:56.476 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 ceph-mds[73582]: -- 172.21.15.39:0/3624801658 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620166e5a40 con 0x5620166b9000 2024-03-16T10:05:56.476 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: starting mds.a at 2024-03-16T10:05:56.476 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 bash[73563]: fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be 2024-03-16T10:05:56.476 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:05:56 smithi039 systemd[1]: Started Ceph mds.a for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 -- 172.21.15.39:0/2069546813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e8010dbb0 msgr2=0x7f7e8010ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 --2- 172.21.15.39:0/2069546813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e8010dbb0 0x7f7e8010ffa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009cd0 tx=0x7f7e7802c0e0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 -- 172.21.15.39:0/2069546813 shutdown_connections 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 --2- 172.21.15.39:0/2069546813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e8010dbb0 0x7f7e8010ffa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 --2- 172.21.15.39:0/2069546813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e800746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 --2- 172.21.15.39:0/2069546813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e80075820 0x7f7e80073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 -- 172.21.15.39:0/2069546813 >> 172.21.15.39:0/2069546813 conn(0x7f7e800fcb70 msgr2=0x7f7e800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 -- 172.21.15.39:0/2069546813 shutdown_connections 2024-03-16T10:05:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.616+0000 7f7e7f200640 1 -- 172.21.15.39:0/2069546813 wait complete. 2024-03-16T10:05:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.617+0000 7f7e7f200640 1 Processor -- start 2024-03-16T10:05:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 -- start start 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e80075820 0x7f7e8019e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e8010dbb0 0x7f7e801a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80113830 con 0x7f7e8010dbb0 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801136b0 con 0x7f7e80075820 2024-03-16T10:05:56.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.618+0000 7f7e7f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801139b0 con 0x7f7e80074240 2024-03-16T10:05:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:39678/0 (socket says 172.21.15.39:39678) 2024-03-16T10:05:56.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 -- 172.21.15.39:0/3985932955 learned_addr learned my addr 172.21.15.39:0/3985932955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7d400640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e80075820 0x7f7e8019e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7e800640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e8010dbb0 0x7f7e801a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 -- 172.21.15.39:0/3985932955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e80075820 msgr2=0x7f7e8019e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e80075820 0x7f7e8019e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 -- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e8010dbb0 msgr2=0x7f7e801a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.619+0000 7f7e7de00640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e8010dbb0 0x7f7e801a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e7de00640 1 -- 172.21.15.39:0/3985932955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801a5aa0 con 0x7f7e80074240 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e7de00640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9f0 tx=0x7f7e7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000cd80 con 0x7f7e80074240 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801a5d90 con 0x7f7e80074240 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e70010910 con 0x7f7e80074240 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700188f0 con 0x7f7e80074240 2024-03-16T10:05:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.620+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a6200 con 0x7f7e80074240 2024-03-16T10:05:56.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.621+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e80103d80 con 0x7f7e80074240 2024-03-16T10:05:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.623+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7e70010ac0 con 0x7f7e80074240 2024-03-16T10:05:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.625+0000 7f7e76a00640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 0x7f7e640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.625+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7e700cea10 con 0x7f7e80074240 2024-03-16T10:05:56.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.625+0000 7f7e7d400640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 0x7f7e640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.625+0000 7f7e7d400640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 0x7f7e640a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005d00 tx=0x7f7e6c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:56.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.629+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e7009beb0 con 0x7f7e80074240 2024-03-16T10:05:56.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.871+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e80061c20 con 0x7f7e80074240 2024-03-16T10:05:56.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.872+0000 7f7e76a00640 1 -- 172.21.15.39:0/3985932955 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f7e7009b7b0 con 0x7f7e80074240 2024-03-16T10:05:56.874 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:56.874 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-03-16T10:05:56.874 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2024-03-16T10:05:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 msgr2=0x7f7e640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 0x7f7e640a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c005d00 tx=0x7f7e6c005c50 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 msgr2=0x7f7e8019de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000e9f0 tx=0x7f7e7000eeb0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 shutdown_connections 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7e640a2800 0x7f7e640a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e8010dbb0 0x7f7e801a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e80075820 0x7f7e8019e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 --2- 172.21.15.39:0/3985932955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e80074240 0x7f7e8019de80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 >> 172.21.15.39:0/3985932955 conn(0x7f7e800fcb70 msgr2=0x7f7e80100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.875+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 shutdown_connections 2024-03-16T10:05:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:56.876+0000 7f7e7f200640 1 -- 172.21.15.39:0/3985932955 wait complete. 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:boot 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: fsmap 11 up:standby 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:57.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:57.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: Deploying daemon mds.b on smithi105 2024-03-16T10:05:57.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:57 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3985932955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:boot 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: fsmap 11 up:standby 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:57.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:57.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:57.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: Deploying daemon mds.b on smithi105 2024-03-16T10:05:57.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:57 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3985932955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:boot 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: fsmap 11 up:standby 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: Deploying daemon mds.b on smithi105 2024-03-16T10:05:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:57 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3985932955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:05:58.792 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:58 smithi105 systemd[1]: Starting Ceph mds.b for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:05:58.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:05:59.043 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:58 smithi105 podman[67199]: 2024-03-16 10:05:58.693306864 +0000 UTC m=+0.017354769 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:05:59.043 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:58 smithi105 podman[67199]: 2024-03-16 10:05:58.806231007 +0000 UTC m=+0.130278903 container create c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:05:59.043 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 podman[67199]: 2024-03-16 10:05:59.017243545 +0000 UTC m=+0.341291444 container init c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T10:05:59.043 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 podman[67199]: 2024-03-16 10:05:59.019874994 +0000 UTC m=+0.343922893 container start c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T10:05:59.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 -- 172.21.15.39:0/3261673358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900761a0 msgr2=0x7fea900765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 --2- 172.21.15.39:0/3261673358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900761a0 0x7fea900765c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea80009900 tx=0x7fea8002bc40 comp rx=0 tx=0).stop 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 -- 172.21.15.39:0/3261673358 shutdown_connections 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 --2- 172.21.15.39:0/3261673358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea90078620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 --2- 172.21.15.39:0/3261673358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900773e0 0x7fea90077840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 --2- 172.21.15.39:0/3261673358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900761a0 0x7fea900765c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 -- 172.21.15.39:0/3261673358 >> 172.21.15.39:0/3261673358 conn(0x7fea900fed10 msgr2=0x7fea90101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 -- 172.21.15.39:0/3261673358 shutdown_connections 2024-03-16T10:05:59.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.132+0000 7fea8fe00640 1 -- 172.21.15.39:0/3261673358 wait complete. 2024-03-16T10:05:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.133+0000 7fea8fe00640 1 Processor -- start 2024-03-16T10:05:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.133+0000 7fea8fe00640 1 -- start start 2024-03-16T10:05:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.133+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 0x7fea9019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea901a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea901138b0 con 0x7fea90077d80 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea90113730 con 0x7fea900773e0 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea90113a30 con 0x7fea900761a0 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55084/0 (socket says 172.21.15.39:55084) 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 -- 172.21.15.39:0/547366556 learned_addr learned my addr 172.21.15.39:0/547366556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8ea00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 0x7fea9019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8f400640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea901a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 -- 172.21.15.39:0/547366556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 msgr2=0x7fea9019ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 0x7fea9019ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 -- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 msgr2=0x7fea901a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea901a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8e000640 1 -- 172.21.15.39:0/547366556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea901a5af0 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8ea00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 0x7fea9019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.134+0000 7fea8f400640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea901a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea8e000640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d770 tx=0x7fea7c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c017070 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea7c005500 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c004e00 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901a5de0 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.135+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a6250 con 0x7fea900773e0 2024-03-16T10:05:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.136+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea90068980 con 0x7fea900773e0 2024-03-16T10:05:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.139+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fea7c004fa0 con 0x7fea900773e0 2024-03-16T10:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.140+0000 7fea94a00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 0x7fea740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.140+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fea7c0ce0b0 con 0x7fea900773e0 2024-03-16T10:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.140+0000 7fea8ea00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 0x7fea740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.141+0000 7fea8ea00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 0x7fea740a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea8002c250 tx=0x7fea80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.144+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea7c09b550 con 0x7fea900773e0 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-mds[67218]: -- 172.21.15.105:0/3197833802 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c29de9fa40 con 0x55c29de73000 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: starting mds.b at 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 bash[67199]: c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 2024-03-16T10:05:59.365 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:05:59 smithi105 systemd[1]: Started Ceph mds.b for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:05:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.395+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea90062480 con 0x7fea900773e0 2024-03-16T10:05:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.396+0000 7fea94a00640 1 -- 172.21.15.39:0/547366556 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fea7c09ae50 con 0x7fea900773e0 2024-03-16T10:05:59.397 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:59.397 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-16T10:05:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2024-03-16T10:05:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 msgr2=0x7fea740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 0x7fea740a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea8002c250 tx=0x7fea80005fd0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 msgr2=0x7fea9019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d770 tx=0x7fea7c00dc30 comp rx=0 tx=0).stop 2024-03-16T10:05:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 shutdown_connections 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fea740a2880 0x7fea740a4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea90077d80 0x7fea901a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea900773e0 0x7fea9019e410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 --2- 172.21.15.39:0/547366556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea900761a0 0x7fea9019ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 >> 172.21.15.39:0/547366556 conn(0x7fea900fed10 msgr2=0x7fea90100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 shutdown_connections 2024-03-16T10:05:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.399+0000 7fea8fe00640 1 -- 172.21.15.39:0/547366556 wait complete. 2024-03-16T10:05:59.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:05:59.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.646+0000 7f0355600640 1 -- 172.21.15.39:0/4287398972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 msgr2=0x7f0350100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.646+0000 7f0355600640 1 --2- 172.21.15.39:0/4287398972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350100b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0340009900 tx=0x7f034002bc30 comp rx=0 tx=0).stop 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 -- 172.21.15.39:0/4287398972 shutdown_connections 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 --2- 172.21.15.39:0/4287398972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 0x7f0350108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 --2- 172.21.15.39:0/4287398972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 0x7f0350101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 --2- 172.21.15.39:0/4287398972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350100b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 -- 172.21.15.39:0/4287398972 >> 172.21.15.39:0/4287398972 conn(0x7f03500fbc70 msgr2=0x7f03500fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 -- 172.21.15.39:0/4287398972 shutdown_connections 2024-03-16T10:05:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.647+0000 7f0355600640 1 -- 172.21.15.39:0/4287398972 wait complete. 2024-03-16T10:05:59.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 Processor -- start 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 -- start start 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 0x7f0350102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 0x7f0350103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f035010c3a0 con 0x7f03501022b0 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f035010c220 con 0x7f0350100700 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.648+0000 7f0355600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f035010c520 con 0x7f03501018f0 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55110/0 (socket says 172.21.15.39:55110) 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 -- 172.21.15.39:0/2754436429 learned_addr learned my addr 172.21.15.39:0/2754436429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034f400640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 0x7f0350103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034e000640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 0x7f0350102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 -- 172.21.15.39:0/2754436429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 msgr2=0x7f0350102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 0x7f0350102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 -- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 msgr2=0x7f0350103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 0x7f0350103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 -- 172.21.15.39:0/2754436429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0350103aa0 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.649+0000 7f034ea00640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f034002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.650+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340043020 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.650+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03400054e0 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.650+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501a1c50 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.650+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340003c40 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.650+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a21a0 con 0x7f0350100700 2024-03-16T10:05:59.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.651+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03501a2370 con 0x7f0350100700 2024-03-16T10:05:59.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.654+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0340046020 con 0x7f0350100700 2024-03-16T10:05:59.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.655+0000 7f0347400640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 0x7f03340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:05:59.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.655+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f03400f05c0 con 0x7f0350100700 2024-03-16T10:05:59.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.655+0000 7f034e000640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 0x7f03340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:05:59.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.656+0000 7f034e000640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 0x7f03340a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0350104400 tx=0x7f033c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:05:59.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.659+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f034004d020 con 0x7f0350100700 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:05:59 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:05:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:05:59 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:05:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.866+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03501a25e0 con 0x7f0350100700 2024-03-16T10:05:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.868+0000 7f0347400640 1 -- 172.21.15.39:0/2754436429 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18445 (secure 0 0 0) 0x7f03400bda60 con 0x7f0350100700 2024-03-16T10:05:59.868 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:05:59.869 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:05:08.845239+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.870+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 msgr2=0x7f03340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.870+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 0x7f03340a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0350104400 tx=0x7f033c005f90 comp rx=0 tx=0).stop 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 msgr2=0x7f0350104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f034002c650 comp rx=0 tx=0).stop 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 shutdown_connections 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f03340a2880 0x7f03340a4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03501022b0 0x7f0350103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03501018f0 0x7f0350102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 --2- 172.21.15.39:0/2754436429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0350100700 0x7f0350104690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:05:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 >> 172.21.15.39:0/2754436429 conn(0x7f03500fbc70 msgr2=0x7f03500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:05:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 shutdown_connections 2024-03-16T10:05:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:05:59.871+0000 7f0355600640 1 -- 172.21.15.39:0/2754436429 wait complete. 2024-03-16T10:05:59.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:05:59.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:06:00.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 -- 172.21.15.39:0/1338805833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 msgr2=0x7f9b9c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/1338805833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c101c10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009ca0 tx=0x7f9b8c02c100 comp rx=0 tx=0).stop 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 -- 172.21.15.39:0/1338805833 shutdown_connections 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/1338805833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/1338805833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/1338805833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c101c10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 -- 172.21.15.39:0/1338805833 >> 172.21.15.39:0/1338805833 conn(0x7f9b9c0fcd80 msgr2=0x7f9b9c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 -- 172.21.15.39:0/1338805833 shutdown_connections 2024-03-16T10:06:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.107+0000 7f9ba3c00640 1 -- 172.21.15.39:0/1338805833 wait complete. 2024-03-16T10:06:00.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 Processor -- start 2024-03-16T10:06:00.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 -- start start 2024-03-16T10:06:00.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10d440 con 0x7f9b9c1017f0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10d2c0 con 0x7f9b9c1033a0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba3c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c10d5c0 con 0x7f9b9c1029e0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40344/0 (socket says 172.21.15.39:40344) 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9b93e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37380/0 (socket says 172.21.15.39:37380) 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.108+0000 7f9ba1200640 1 -- 172.21.15.39:0/2720696257 learned_addr learned my addr 172.21.15.39:0/2720696257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 -- 172.21.15.39:0/2720696257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 msgr2=0x7f9b9c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 -- 172.21.15.39:0/2720696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 msgr2=0x7f9b9c074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 -- 172.21.15.39:0/2720696257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0749a0 con 0x7f9b9c1017f0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9ba1200640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f80 tx=0x7f9b8c035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.109+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c039040 con 0x7f9b9c1017f0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.110+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c1a1c20 con 0x7f9b9c1017f0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.110+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c044070 con 0x7f9b9c1017f0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.110+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0044a0 con 0x7f9b9c1017f0 2024-03-16T10:06:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.110+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c1a2190 con 0x7f9b9c1017f0 2024-03-16T10:06:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.111+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c108be0 con 0x7f9b9c1017f0 2024-03-16T10:06:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.113+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9b8c004640 con 0x7f9b9c1017f0 2024-03-16T10:06:00.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.114+0000 7f9b9a000640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 0x7f9b800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.114+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9b8c0f16b0 con 0x7f9b9c1017f0 2024-03-16T10:06:00.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.114+0000 7f9b93e00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 0x7f9b800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.115+0000 7f9b93e00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 0x7f9b800a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b840095f0 tx=0x7f9b84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.117+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9b8c0beb50 con 0x7f9b9c1017f0 2024-03-16T10:06:00.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.353+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b9c0623e0 con 0x7f9b9c1017f0 2024-03-16T10:06:00.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.353+0000 7f9b9a000640 1 -- 172.21.15.39:0/2720696257 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f9b8c0be450 con 0x7f9b9c1017f0 2024-03-16T10:06:00.354 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:06:00.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 msgr2=0x7f9b800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 0x7f9b800a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b840095f0 tx=0x7f9b84005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:00.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 msgr2=0x7f9b9c075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f80 tx=0x7f9b8c035d10 comp rx=0 tx=0).stop 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 shutdown_connections 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9b800a2800 0x7f9b800a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b9c1033a0 0x7f9b9c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b9c1029e0 0x7f9b9c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 --2- 172.21.15.39:0/2720696257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b9c1017f0 0x7f9b9c075590 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 >> 172.21.15.39:0/2720696257 conn(0x7f9b9c0fcd80 msgr2=0x7f9b9c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 shutdown_connections 2024-03-16T10:06:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.356+0000 7f9ba3c00640 1 -- 172.21.15.39:0/2720696257 wait complete. 2024-03-16T10:06:00.375 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T10:06:00.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 -- 172.21.15.39:0/3031225019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178108f70 msgr2=0x7fa1781093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 --2- 172.21.15.39:0/3031225019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178108f70 0x7fa1781093f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa164009900 tx=0x7fa16402bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 -- 172.21.15.39:0/3031225019 shutdown_connections 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 --2- 172.21.15.39:0/3031225019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178109930 0x7fa178110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 --2- 172.21.15.39:0/3031225019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178108f70 0x7fa1781093f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 --2- 172.21.15.39:0/3031225019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178107e20 0x7fa178108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 -- 172.21.15.39:0/3031225019 >> 172.21.15.39:0/3031225019 conn(0x7fa178076df0 msgr2=0x7fa178079210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 -- 172.21.15.39:0/3031225019 shutdown_connections 2024-03-16T10:06:00.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.558+0000 7fa17f600640 1 -- 172.21.15.39:0/3031225019 wait complete. 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 Processor -- start 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 -- start start 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178107e20 0x7fa17819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 0x7fa17819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 0x7fa1781a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa178113b20 con 0x7fa178108f70 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781139a0 con 0x7fa178109930 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17f600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa178113ca0 con 0x7fa178107e20 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa177e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 0x7fa17819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 0x7fa1781a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa177e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 0x7fa17819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40374/0 (socket says 172.21.15.39:40374) 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa177e00640 1 -- 172.21.15.39:0/3325735366 learned_addr learned my addr 172.21.15.39:0/3325735366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.559+0000 7fa17cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178107e20 0x7fa17819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 -- 172.21.15.39:0/3325735366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178107e20 msgr2=0x7fa17819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178107e20 0x7fa17819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 -- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 msgr2=0x7fa17819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 0x7fa17819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 -- 172.21.15.39:0/3325735366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1781a5d50 con 0x7fa178109930 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17d600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 0x7fa1781a56e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa170009fd0 tx=0x7fa17000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000ce20 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa170005020 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170010720 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1781a5fe0 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.560+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781a6500 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.561+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17810f0d0 con 0x7fa178109930 2024-03-16T10:06:00.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.563+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa170005540 con 0x7fa178109930 2024-03-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.564+0000 7fa175600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 0x7fa1600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.565+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1700cec90 con 0x7fa178109930 2024-03-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.565+0000 7fa17cc00640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 0x7fa1600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.565+0000 7fa17cc00640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 0x7fa1600a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0095f0 tx=0x7fa16c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:00.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.569+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa17009c130 con 0x7fa178109930 2024-03-16T10:06:00.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:boot 2024-03-16T10:06:00.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: fsmap 12 up:standby 2024-03-16T10:06:00.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:06:00.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/547366556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.574 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:00 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2754436429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:boot 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: fsmap 12 up:standby 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/547366556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:00 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2754436429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.746 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:boot 2024-03-16T10:06:00.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: fsmap 12 up:standby 2024-03-16T10:06:00.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:06:00.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/547366556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:00 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2754436429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.775+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa1780623e0 con 0x7fa178109930 2024-03-16T10:06:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.776+0000 7fa175600640 1 -- 172.21.15.39:0/3325735366 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fa17009ba30 con 0x7fa178109930 2024-03-16T10:06:00.776 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:00.776 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-03-16T10:06:00.776 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:06:00.776 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-03-16T10:06:00.777 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-03-16T10:06:00.777 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:06:00.777 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-03-16T10:06:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.778+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 msgr2=0x7fa1600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.778+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 0x7fa1600a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0095f0 tx=0x7fa16c005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.778+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 msgr2=0x7fa1781a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.778+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 0x7fa1781a56e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa170009fd0 tx=0x7fa17000ef10 comp rx=0 tx=0).stop 2024-03-16T10:06:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 shutdown_connections 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa1600a2800 0x7fa1600a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa178109930 0x7fa1781a56e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa178108f70 0x7fa17819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 --2- 172.21.15.39:0/3325735366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa178107e20 0x7fa17819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 >> 172.21.15.39:0/3325735366 conn(0x7fa178076df0 msgr2=0x7fa17810c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 shutdown_connections 2024-03-16T10:06:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:00.779+0000 7fa17f600640 1 -- 172.21.15.39:0/3325735366 wait complete. 2024-03-16T10:06:00.798 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T10:06:00.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 -- 172.21.15.39:0/3117532806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 msgr2=0x7f8be4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 --2- 172.21.15.39:0/3117532806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be4101d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009900 tx=0x7f8bcc02bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 -- 172.21.15.39:0/3117532806 shutdown_connections 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 --2- 172.21.15.39:0/3117532806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 0x7f8be4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 --2- 172.21.15.39:0/3117532806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be4101d70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 --2- 172.21.15.39:0/3117532806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 -- 172.21.15.39:0/3117532806 >> 172.21.15.39:0/3117532806 conn(0x7f8be40fbc70 msgr2=0x7f8be40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 -- 172.21.15.39:0/3117532806 shutdown_connections 2024-03-16T10:06:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.026+0000 7f8beaa00640 1 -- 172.21.15.39:0/3117532806 wait complete. 2024-03-16T10:06:01.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.027+0000 7f8beaa00640 1 Processor -- start 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.027+0000 7f8beaa00640 1 -- start start 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 0x7f8be41a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8be410c360 con 0x7f8be4100700 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8be410c1e0 con 0x7f8be41018f0 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8beaa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8be410c4e0 con 0x7f8be41022b0 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40388/0 (socket says 172.21.15.39:40388) 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be3e00640 1 -- 172.21.15.39:0/138298059 learned_addr learned my addr 172.21.15.39:0/138298059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8bdb400640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be8a00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 0x7f8be41a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be3e00640 1 -- 172.21.15.39:0/138298059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 msgr2=0x7f8be41a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.028+0000 7f8be3e00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 0x7f8be41a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be3e00640 1 -- 172.21.15.39:0/138298059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 msgr2=0x7f8be419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be3e00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be3e00640 1 -- 172.21.15.39:0/138298059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be41a1a30 con 0x7f8be4100700 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be3e00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ebc0 tx=0x7f8bd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4003b70 con 0x7f8be4100700 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd4003d10 con 0x7f8be4100700 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41a1d20 con 0x7f8be4100700 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4019980 con 0x7f8be4100700 2024-03-16T10:06:01.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.029+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41a2190 con 0x7f8be4100700 2024-03-16T10:06:01.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.030+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be41a2420 con 0x7f8be4100700 2024-03-16T10:06:01.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.032+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8bd4019b20 con 0x7f8be4100700 2024-03-16T10:06:01.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.034+0000 7f8be1600640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 0x7f8bc80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:01.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.034+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8bd40cef30 con 0x7f8be4100700 2024-03-16T10:06:01.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.034+0000 7f8bdb400640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 0x7f8bc80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:01.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.035+0000 7f8bdb400640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 0x7f8bc80a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:01.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.036+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8bd409c460 con 0x7f8be4100700 2024-03-16T10:06:01.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:01.295+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8be41a2760 con 0x7f8be4100700 2024-03-16T10:06:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2720696257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3325735366' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:01 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.196+0000 7f8be1600640 1 -- 172.21.15.39:0/138298059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f8bd409bd60 con 0x7f8be4100700 2024-03-16T10:06:02.197 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-03-16T10:06:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.199+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 msgr2=0x7f8bc80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.199+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 0x7f8bc80a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009230 tx=0x7f8bcc005fd0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.199+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 msgr2=0x7f8be4199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.199+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ebc0 tx=0x7f8bd400c6a0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 shutdown_connections 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8bc80a2800 0x7f8bc80a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be41022b0 0x7f8be41a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be41018f0 0x7f8be419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 --2- 172.21.15.39:0/138298059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8be4100700 0x7f8be4199e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 >> 172.21.15.39:0/138298059 conn(0x7f8be40fbc70 msgr2=0x7f8be4069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 shutdown_connections 2024-03-16T10:06:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.200+0000 7f8beaa00640 1 -- 172.21.15.39:0/138298059 wait complete. 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2720696257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3325735366' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:01 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.224 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:02.224 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2720696257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:02.224 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3325735366' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T10:06:02.224 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.225 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:01 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:02.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-16T10:06:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.420+0000 7ff4f3e00640 1 -- 172.21.15.39:0/2032220286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f4101290 msgr2=0x7ff4f4101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.420+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/2032220286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f4101290 0x7ff4f4101710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009be0 tx=0x7ff4ec02bef0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.420+0000 7ff4f3e00640 1 -- 172.21.15.39:0/2032220286 shutdown_connections 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/2032220286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f4101290 0x7ff4f4101710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/2032220286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f41008f0 0x7ff4f4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/2032220286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 0x7ff4f41003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 -- 172.21.15.39:0/2032220286 >> 172.21.15.39:0/2032220286 conn(0x7ff4f40fbb50 msgr2=0x7ff4f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 -- 172.21.15.39:0/2032220286 shutdown_connections 2024-03-16T10:06:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.421+0000 7ff4f3e00640 1 -- 172.21.15.39:0/2032220286 wait complete. 2024-03-16T10:06:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 Processor -- start 2024-03-16T10:06:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 -- start start 2024-03-16T10:06:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 0x7ff4f4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410f580 con 0x7ff4f40fff90 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410f400 con 0x7ff4f41008f0 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.422+0000 7ff4f3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f410f700 con 0x7ff4f4101290 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 0x7ff4f4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37422/0 (socket says 172.21.15.39:37422) 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f3400640 1 -- 172.21.15.39:0/850055778 learned_addr learned my addr 172.21.15.39:0/850055778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55816/0 (socket says 172.21.15.39:55816) 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 -- 172.21.15.39:0/850055778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 msgr2=0x7ff4f41a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 -- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 msgr2=0x7ff4f4199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 0x7ff4f4199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 -- 172.21.15.39:0/850055778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f41a1760 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f3400640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4f2000640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9f0 tx=0x7ff4e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.423+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000ccc0 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.424+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e0005020 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.424+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41a1a50 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.424+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004a00 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.424+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41a1fa0 con 0x7ff4f41008f0 2024-03-16T10:06:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.425+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f410adf0 con 0x7ff4f41008f0 2024-03-16T10:06:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.427+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff4e00051d0 con 0x7ff4f41008f0 2024-03-16T10:06:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.428+0000 7ff4eb400640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 0x7ff4d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.429+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7ff4e00ce690 con 0x7ff4f41008f0 2024-03-16T10:06:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.429+0000 7ff4f2a00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 0x7ff4d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.429+0000 7ff4f2a00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 0x7ff4d80a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4005fd0 tx=0x7ff4e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.433+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff4e009b9c0 con 0x7ff4f41008f0 2024-03-16T10:06:02.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:02.638+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff4f402d1b0 con 0x7ff4f41008f0 2024-03-16T10:06:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.214+0000 7ff4eb400640 1 -- 172.21.15.39:0/850055778 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7ff4e009b2c0 con 0x7ff4f41008f0 2024-03-16T10:06:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-03-16T10:06:03.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.222+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 msgr2=0x7ff4d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:03.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.222+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 0x7ff4d80a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4005fd0 tx=0x7ff4e4009260 comp rx=0 tx=0).stop 2024-03-16T10:06:03.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.222+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 msgr2=0x7ff4f419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:03.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.222+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9f0 tx=0x7ff4e000edf0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 shutdown_connections 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff4d80a2880 0x7ff4d80a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4f4101290 0x7ff4f41a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4f41008f0 0x7ff4f419a080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 --2- 172.21.15.39:0/850055778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f40fff90 0x7ff4f4199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.223+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 >> 172.21.15.39:0/850055778 conn(0x7ff4f40fbb50 msgr2=0x7ff4f40fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.224+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 shutdown_connections 2024-03-16T10:06:03.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.224+0000 7ff4f3e00640 1 -- 172.21.15.39:0/850055778 wait complete. 2024-03-16T10:06:03.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T10:06:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T10:06:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T10:06:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T10:06:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:03 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 -- 172.21.15.39:0/4238584801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 msgr2=0x7f5508106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 --2- 172.21.15.39:0/4238584801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508106e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009860 tx=0x7f54f402bb70 comp rx=0 tx=0).stop 2024-03-16T10:06:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 -- 172.21.15.39:0/4238584801 shutdown_connections 2024-03-16T10:06:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 --2- 172.21.15.39:0/4238584801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 0x7f55081097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 --2- 172.21.15.39:0/4238584801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508106e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 --2- 172.21.15.39:0/4238584801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f5508104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 -- 172.21.15.39:0/4238584801 >> 172.21.15.39:0/4238584801 conn(0x7f55080fbb50 msgr2=0x7f55080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.457+0000 7f5507e00640 1 -- 172.21.15.39:0/4238584801 shutdown_connections 2024-03-16T10:06:03.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.458+0000 7f5507e00640 1 -- 172.21.15.39:0/4238584801 wait complete. 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.458+0000 7f5507e00640 1 Processor -- start 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 -- start start 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 0x7f550819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d340 con 0x7f5508102140 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d1c0 con 0x7f5508104a70 2024-03-16T10:06:03.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5507e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f550810d4c0 con 0x7f55081073a0 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5506a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5506a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40424/0 (socket says 172.21.15.39:40424) 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5506a00640 1 -- 172.21.15.39:0/3377502548 learned_addr learned my addr 172.21.15.39:0/3377502548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.459+0000 7f5506000640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5507400640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 0x7f550819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 -- 172.21.15.39:0/3377502548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 msgr2=0x7f550819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 0x7f550819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 -- 172.21.15.39:0/3377502548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 msgr2=0x7f5508197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 -- 172.21.15.39:0/3377502548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550819f600 con 0x7f5508102140 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506000640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:03.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f5506a00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9c0 tx=0x7f54f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.460+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f800cde0 con 0x7f5508102140 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.461+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550819f8f0 con 0x7f5508102140 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.461+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f80035f0 con 0x7f5508102140 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.461+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8010650 con 0x7f5508102140 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.461+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550819fe60 con 0x7f5508102140 2024-03-16T10:06:03.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.462+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55080fd640 con 0x7f5508102140 2024-03-16T10:06:03.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.466+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f54f80037a0 con 0x7f5508102140 2024-03-16T10:06:03.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.467+0000 7f550ca00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 0x7f54ec0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:03.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.467+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f54f80cec00 con 0x7f5508102140 2024-03-16T10:06:03.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.467+0000 7f5506000640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 0x7f54ec0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:03.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.468+0000 7f5506000640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 0x7f54ec0a4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54f40349e0 tx=0x7f54f402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:03.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.472+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f54f809bdb0 con 0x7f5508102140 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:03 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/138298059' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:03.509 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:03.510 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:03 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:06:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:03.732+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f550802d1b0 con 0x7f5508102140 2024-03-16T10:06:04.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.229+0000 7f550ca00640 1 -- 172.21.15.39:0/3377502548 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f54f80bb070 con 0x7f5508102140 2024-03-16T10:06:04.230 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T10:06:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.235+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 msgr2=0x7f54ec0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.235+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 0x7f54ec0a4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f54f40349e0 tx=0x7f54f402e040 comp rx=0 tx=0).stop 2024-03-16T10:06:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.235+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 msgr2=0x7f55081979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.235+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9c0 tx=0x7f54f800ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:04.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 shutdown_connections 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f54ec0a2900 0x7f54ec0a4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f55081073a0 0x7f550819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5508104a70 0x7f5508197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 --2- 172.21.15.39:0/3377502548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508102140 0x7f55081979e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 >> 172.21.15.39:0/3377502548 conn(0x7f55080fbb50 msgr2=0x7f5508102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.236+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 shutdown_connections 2024-03-16T10:06:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.237+0000 7f5507e00640 1 -- 172.21.15.39:0/3377502548 wait complete. 2024-03-16T10:06:04.261 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-16T10:06:04.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-16T10:06:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.454+0000 7feb7de00640 1 -- 172.21.15.39:0/1796116831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 msgr2=0x7feb78105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.454+0000 7feb7de00640 1 --2- 172.21.15.39:0/1796116831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb78105090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb64009900 tx=0x7feb6402bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 -- 172.21.15.39:0/1796116831 shutdown_connections 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 --2- 172.21.15.39:0/1796116831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb7810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 --2- 172.21.15.39:0/1796116831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb78105090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 --2- 172.21.15.39:0/1796116831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 -- 172.21.15.39:0/1796116831 >> 172.21.15.39:0/1796116831 conn(0x7feb780fef90 msgr2=0x7feb781013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 -- 172.21.15.39:0/1796116831 shutdown_connections 2024-03-16T10:06:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.455+0000 7feb7de00640 1 -- 172.21.15.39:0/1796116831 wait complete. 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 Processor -- start 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 -- start start 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb7819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb781a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb7810f6a0 con 0x7feb781055d0 2024-03-16T10:06:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb7810f520 con 0x7feb78103a20 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb7de00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb7810f820 con 0x7feb78104c10 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb77400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb781a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb76a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb76a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55874/0 (socket says 172.21.15.39:55874) 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb76a00640 1 -- 172.21.15.39:0/2644892031 learned_addr learned my addr 172.21.15.39:0/2644892031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb77400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb781a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40450/0 (socket says 172.21.15.39:40450) 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.456+0000 7feb76000640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb7819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 -- 172.21.15.39:0/2644892031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 msgr2=0x7feb7819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb7819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 -- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 msgr2=0x7feb781a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb781a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 -- 172.21.15.39:0/2644892031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb781a1990 con 0x7feb78103a20 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb76a00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb6800e9c0 tx=0x7feb6800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800cd80 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb680035f0 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781a1c80 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68010690 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.457+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781a21a0 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.459+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7810ad60 con 0x7feb78103a20 2024-03-16T10:06:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.460+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7feb68004030 con 0x7feb78103a20 2024-03-16T10:06:04.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.461+0000 7feb7ca00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 0x7feb5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:04.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.461+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7feb680ceca0 con 0x7feb78103a20 2024-03-16T10:06:04.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.462+0000 7feb76000640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 0x7feb5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:04.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.462+0000 7feb76000640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 0x7feb5c0a4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb6402c7b0 tx=0x7feb64005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.465+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb6809bdf0 con 0x7feb78103a20 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/850055778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:04 smithi039 ceph-mon[31510]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T10:06:04.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:04.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/850055778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-16T10:06:04.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T10:06:04.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T10:06:04.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T10:06:04.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:04.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:04 smithi105 ceph-mon[36559]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T10:06:04.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:04.678+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7feb7802d1b0 con 0x7feb78103a20 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/850055778' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T10:06:04.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T10:06:04.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T10:06:04.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:04.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:04 smithi194 ceph-mon[36445]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T10:06:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.205+0000 7feb7ca00640 1 -- 172.21.15.39:0/2644892031 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7feb6809b6f0 con 0x7feb78103a20 2024-03-16T10:06:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.210+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 msgr2=0x7feb5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.210+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 0x7feb5c0a4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb6402c7b0 tx=0x7feb64005fd0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.210+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 msgr2=0x7feb78199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.210+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feb6800e9c0 tx=0x7feb6800ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 shutdown_connections 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7feb5c0a2880 0x7feb5c0a4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb781055d0 0x7feb781a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb78104c10 0x7feb7819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 --2- 172.21.15.39:0/2644892031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb78103a20 0x7feb78199d70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 >> 172.21.15.39:0/2644892031 conn(0x7feb780fef90 msgr2=0x7feb78107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 shutdown_connections 2024-03-16T10:06:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.211+0000 7feb7de00640 1 -- 172.21.15.39:0/2644892031 wait complete. 2024-03-16T10:06:05.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 -- 172.21.15.39:0/1510173023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 msgr2=0x7fc024108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 --2- 172.21.15.39:0/1510173023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc024108060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc014009900 tx=0x7fc01402bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 -- 172.21.15.39:0/1510173023 shutdown_connections 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 --2- 172.21.15.39:0/1510173023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc024110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 --2- 172.21.15.39:0/1510173023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc0241092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 --2- 172.21.15.39:0/1510173023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc024108060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 -- 172.21.15.39:0/1510173023 >> 172.21.15.39:0/1510173023 conn(0x7fc024076bf0 msgr2=0x7fc024079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 -- 172.21.15.39:0/1510173023 shutdown_connections 2024-03-16T10:06:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.446+0000 7fc023200640 1 -- 172.21.15.39:0/1510173023 wait complete. 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.447+0000 7fc023200640 1 Processor -- start 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 -- start start 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc02419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc0241a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc024113900 con 0x7fc0241097f0 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc024113780 con 0x7fc024107c40 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc023200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc024113a80 con 0x7fc024108e30 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc021e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc021e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55878/0 (socket says 172.21.15.39:55878) 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc021e00640 1 -- 172.21.15.39:0/2845254927 learned_addr learned my addr 172.21.15.39:0/2845254927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc022800640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc0241a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.448+0000 7fc021400640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc02419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 -- 172.21.15.39:0/2845254927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 msgr2=0x7fc02419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc02419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 -- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 msgr2=0x7fc0241a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc0241a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 -- 172.21.15.39:0/2845254927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241a5b90 con 0x7fc024107c40 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021400640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc02419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc022800640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc0241a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc021e00640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc01402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014043020 con 0x7fc024107c40 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0140054e0 con 0x7fc024107c40 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014003ce0 con 0x7fc024107c40 2024-03-16T10:06:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.449+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0241a5e20 con 0x7fc024107c40 2024-03-16T10:06:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.450+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a6230 con 0x7fc024107c40 2024-03-16T10:06:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.451+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02410ef60 con 0x7fc024107c40 2024-03-16T10:06:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.452+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc014046020 con 0x7fc024107c40 2024-03-16T10:06:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.454+0000 7fc01aa00640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 0x7fc0080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.454+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fc0140f0cb0 con 0x7fc024107c40 2024-03-16T10:06:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.454+0000 7fc021400640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 0x7fc0080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.455+0000 7fc021400640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 0x7fc0080a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc02419f480 tx=0x7fc010005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.458+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc01404d020 con 0x7fc024107c40 2024-03-16T10:06:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: pgmap v311: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2644892031' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-16T10:06:05.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:05 smithi039 ceph-mon[31510]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: pgmap v311: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2644892031' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-16T10:06:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:05 smithi105 ceph-mon[36559]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T10:06:05.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:05.669+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc024062530 con 0x7fc024107c40 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: pgmap v311: 97 pgs: 15 creating+peering, 4 active+clean, 78 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3377502548' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T10:06:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2644892031' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-16T10:06:05.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-16T10:06:05.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:05 smithi194 ceph-mon[36445]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T10:06:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:active 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2845254927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:06.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:06 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:06.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:active 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2845254927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:06.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:06 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] up:active 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:06.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:06:06.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2845254927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:06.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:06 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T10:06:07.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.303+0000 7fc01aa00640 1 -- 172.21.15.39:0/2845254927 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fc0140bdcc0 con 0x7fc024107c40 2024-03-16T10:06:07.304 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data_ec' created 2024-03-16T10:06:07.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.310+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 msgr2=0x7fc0080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.310+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 0x7fc0080a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc02419f480 tx=0x7fc010005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.310+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 msgr2=0x7fc02419df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.310+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc01402c6c0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 shutdown_connections 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc0080a2880 0x7fc0080a4d40 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc02419f480 tx=0x7fc010005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0241097f0 0x7fc0241a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc024108e30 0x7fc02419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 --2- 172.21.15.39:0/2845254927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc024107c40 0x7fc02419df70 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc01402c6c0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.311+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 >> 172.21.15.39:0/2845254927 conn(0x7fc024076bf0 msgr2=0x7fc02410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:07.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.314+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 shutdown_connections 2024-03-16T10:06:07.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.315+0000 7fc023200640 1 -- 172.21.15.39:0/2845254927 wait complete. 2024-03-16T10:06:07.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 -- 172.21.15.39:0/1725162938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa018104be0 msgr2=0x7fa018105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 --2- 172.21.15.39:0/1725162938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa018104be0 0x7fa018105060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa000009900 tx=0x7fa00002bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 -- 172.21.15.39:0/1725162938 shutdown_connections 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 --2- 172.21.15.39:0/1725162938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 0x7fa01810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 --2- 172.21.15.39:0/1725162938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa018104be0 0x7fa018105060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 --2- 172.21.15.39:0/1725162938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0181039f0 0x7fa018103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 -- 172.21.15.39:0/1725162938 >> 172.21.15.39:0/1725162938 conn(0x7fa0180fefa0 msgr2=0x7fa0181013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:07.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 -- 172.21.15.39:0/1725162938 shutdown_connections 2024-03-16T10:06:07.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.528+0000 7fa01fa00640 1 -- 172.21.15.39:0/1725162938 wait complete. 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.529+0000 7fa01fa00640 1 Processor -- start 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.529+0000 7fa01fa00640 1 -- start start 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa018104be0 0x7fa01819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 0x7fa0181a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810f6c0 con 0x7fa0181055a0 2024-03-16T10:06:07.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810f540 con 0x7fa0181039f0 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810f840 con 0x7fa018104be0 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55896/0 (socket says 172.21.15.39:55896) 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01d000640 1 -- 172.21.15.39:0/3830146490 learned_addr learned my addr 172.21.15.39:0/3830146490 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa01da00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 0x7fa0181a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.530+0000 7fa017e00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa018104be0 0x7fa01819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 -- 172.21.15.39:0/3830146490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa018104be0 msgr2=0x7fa01819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa018104be0 0x7fa01819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 -- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 msgr2=0x7fa0181a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 0x7fa0181a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:07.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 -- 172.21.15.39:0/3830146490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0181a1940 con 0x7fa0181039f0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01d000640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa00800e9c0 tx=0x7fa00800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00800cd80 con 0x7fa0181039f0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0080035f0 con 0x7fa0181039f0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0181a1c30 con 0x7fa0181039f0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008010690 con 0x7fa0181039f0 2024-03-16T10:06:07.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.531+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a2130 con 0x7fa0181039f0 2024-03-16T10:06:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.532+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01810ad10 con 0x7fa0181039f0 2024-03-16T10:06:07.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.534+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa0080037a0 con 0x7fa0181039f0 2024-03-16T10:06:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.535+0000 7fa015600640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 0x7f9ffc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.536+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fa0080cf0d0 con 0x7fa0181039f0 2024-03-16T10:06:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.536+0000 7fa017e00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 0x7f9ffc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:07.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.536+0000 7fa017e00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 0x7f9ffc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa00002c240 tx=0x7fa00002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:07.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.539+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa00809bef0 con 0x7fa0181039f0 2024-03-16T10:06:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:07 smithi105 ceph-mon[36559]: pgmap v313: 97 pgs: 15 creating+peering, 11 active+clean, 71 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-03-16T10:06:07.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:07 smithi105 ceph-mon[36559]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T10:06:07.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:07 smithi194 ceph-mon[36445]: pgmap v313: 97 pgs: 15 creating+peering, 11 active+clean, 71 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-03-16T10:06:07.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:07 smithi194 ceph-mon[36445]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T10:06:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:07.734+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa01802d1b0 con 0x7fa0181039f0 2024-03-16T10:06:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:07 smithi039 ceph-mon[31510]: pgmap v313: 97 pgs: 15 creating+peering, 11 active+clean, 71 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-03-16T10:06:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:07 smithi039 ceph-mon[31510]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T10:06:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.312+0000 7fa015600640 1 -- 172.21.15.39:0/3830146490 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fa00809b7f0 con 0x7fa0181039f0 2024-03-16T10:06:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:set pool 4 allow_ec_overwrites to true 2024-03-16T10:06:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 msgr2=0x7f9ffc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 0x7f9ffc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa00002c240 tx=0x7fa00002df90 comp rx=0 tx=0).stop 2024-03-16T10:06:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 msgr2=0x7fa018199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa00800e9c0 tx=0x7fa00800ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 shutdown_connections 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ffc0a2800 0x7f9ffc0a4cc0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa00002c240 tx=0x7fa00002df90 comp rx=0 tx=0).stop 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0181055a0 0x7fa0181a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa018104be0 0x7fa01819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 --2- 172.21.15.39:0/3830146490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa0181039f0 0x7fa018199d30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.316+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 >> 172.21.15.39:0/3830146490 conn(0x7fa0180fefa0 msgr2=0x7fa018107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:08.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.318+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 shutdown_connections 2024-03-16T10:06:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.319+0000 7fa01fa00640 1 -- 172.21.15.39:0/3830146490 wait complete. 2024-03-16T10:06:08.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-16T10:06:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 -- 172.21.15.39:0/1810025170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 msgr2=0x7f5dc4077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 --2- 172.21.15.39:0/1810025170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc4077840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009900 tx=0x7f5dac02bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 -- 172.21.15.39:0/1810025170 shutdown_connections 2024-03-16T10:06:08.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 --2- 172.21.15.39:0/1810025170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc4077d80 0x7f5dc4078620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 --2- 172.21.15.39:0/1810025170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc4077840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 --2- 172.21.15.39:0/1810025170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc40761a0 0x7f5dc40765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.541+0000 7f5dc3200640 1 -- 172.21.15.39:0/1810025170 >> 172.21.15.39:0/1810025170 conn(0x7f5dc40fed60 msgr2=0x7f5dc41011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.542+0000 7f5dc3200640 1 -- 172.21.15.39:0/1810025170 shutdown_connections 2024-03-16T10:06:08.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.542+0000 7f5dc3200640 1 -- 172.21.15.39:0/1810025170 wait complete. 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 Processor -- start 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 -- start start 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 0x7f5dc419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc4077d80 0x7f5dc41a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc4113840 con 0x7f5dc40773e0 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc41136c0 con 0x7f5dc40761a0 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc3200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc41139c0 con 0x7f5dc4077d80 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 0x7f5dc419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40508/0 (socket says 172.21.15.39:40508) 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.543+0000 7f5dbbe00640 1 -- 172.21.15.39:0/3204375937 learned_addr learned my addr 172.21.15.39:0/3204375937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc4077d80 0x7f5dc41a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 -- 172.21.15.39:0/3204375937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc4077d80 msgr2=0x7f5dc41a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc4077d80 0x7f5dc41a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 -- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 msgr2=0x7f5dc419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 -- 172.21.15.39:0/3204375937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41a5b70 con 0x7f5dc40761a0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dbbe00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 0x7f5dc419df50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e9f0 tx=0x7f5db400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db400cd80 con 0x7f5dc40761a0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db40035f0 con 0x7f5dc40761a0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc41a5e60 con 0x7f5dc40761a0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.544+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4010690 con 0x7f5dc40761a0 2024-03-16T10:06:08.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.545+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc41a62c0 con 0x7f5dc40761a0 2024-03-16T10:06:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.546+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc4110160 con 0x7f5dc40761a0 2024-03-16T10:06:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.547+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f5db4003b30 con 0x7f5dc40761a0 2024-03-16T10:06:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.549+0000 7f5dbaa00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 0x7f5da80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.549+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f5db40ceff0 con 0x7f5dc40761a0 2024-03-16T10:06:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.549+0000 7f5dbbe00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 0x7f5da80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.549+0000 7f5dbbe00640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 0x7f5da80a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5dc419f460 tx=0x7f5dac02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:08.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.553+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5db409be70 con 0x7f5dc40761a0 2024-03-16T10:06:08.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:08 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T10:06:08.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:08 smithi105 ceph-mon[36559]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T10:06:08.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:08 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3830146490' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:08 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:08 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T10:06:08.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:08 smithi194 ceph-mon[36445]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T10:06:08.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:08 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3830146490' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:08 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:08 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T10:06:08.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:08 smithi039 ceph-mon[31510]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T10:06:08.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:08 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3830146490' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:08 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-16T10:06:08.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:08.796+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f5dc402d1b0 con 0x7f5dc40761a0 2024-03-16T10:06:09.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.334+0000 7f5dbaa00640 1 -- 172.21.15.39:0/3204375937 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f5db409b770 con 0x7f5dc40761a0 2024-03-16T10:06:09.335 INFO:teuthology.orchestra.run.smithi039.stderr:added data pool 4 to fsmap 2024-03-16T10:06:09.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.338+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 msgr2=0x7f5da80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 0x7f5da80a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5dc419f460 tx=0x7f5dac02c880 comp rx=0 tx=0).stop 2024-03-16T10:06:09.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 msgr2=0x7f5dc419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 0x7f5dc419df50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5db400e9f0 tx=0x7f5db400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 shutdown_connections 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5da80a2800 0x7f5da80a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dc4077d80 0x7f5dc41a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dc40773e0 0x7f5dc419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 --2- 172.21.15.39:0/3204375937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dc40761a0 0x7f5dc419df50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 >> 172.21.15.39:0/3204375937 conn(0x7f5dc40fed60 msgr2=0x7f5dc4100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 shutdown_connections 2024-03-16T10:06:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.339+0000 7f5dc3200640 1 -- 172.21.15.39:0/3204375937 wait complete. 2024-03-16T10:06:09.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 -- 172.21.15.39:0/3862626731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c108e10 msgr2=0x7fe76c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 --2- 172.21.15.39:0/3862626731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c108e10 0x7fe76c109290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe758009900 tx=0x7fe75802bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 -- 172.21.15.39:0/3862626731 shutdown_connections 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 --2- 172.21.15.39:0/3862626731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 --2- 172.21.15.39:0/3862626731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c108e10 0x7fe76c109290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 --2- 172.21.15.39:0/3862626731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c107c20 0x7fe76c108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 -- 172.21.15.39:0/3862626731 >> 172.21.15.39:0/3862626731 conn(0x7fe76c076bb0 msgr2=0x7fe76c078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:09.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 -- 172.21.15.39:0/3862626731 shutdown_connections 2024-03-16T10:06:09.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.574+0000 7fe772c00640 1 -- 172.21.15.39:0/3862626731 wait complete. 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.575+0000 7fe772c00640 1 Processor -- start 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.575+0000 7fe772c00640 1 -- start start 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c108e10 0x7fe76c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c1a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1138d0 con 0x7fe76c1097d0 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c113750 con 0x7fe76c107c20 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c113a50 con 0x7fe76c108e10 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:09.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55920/0 (socket says 172.21.15.39:55920) 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 -- 172.21.15.39:0/66499812 learned_addr learned my addr 172.21.15.39:0/66499812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772200640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c1a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe770e00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c108e10 0x7fe76c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 -- 172.21.15.39:0/66499812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c108e10 msgr2=0x7fe76c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c108e10 0x7fe76c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 -- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 msgr2=0x7fe76c1a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c1a5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe771800640 1 -- 172.21.15.39:0/66499812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c1a5b00 con 0x7fe76c107c20 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.576+0000 7fe772200640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c1a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe771800640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9c0 tx=0x7fe76000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76000cd80 con 0x7fe76c107c20 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7600035f0 con 0x7fe76c107c20 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760010690 con 0x7fe76c107c20 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c1a5df0 con 0x7fe76c107c20 2024-03-16T10:06:09.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.577+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c1a61a0 con 0x7fe76c107c20 2024-03-16T10:06:09.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.578+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76c10ef40 con 0x7fe76c107c20 2024-03-16T10:06:09.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.580+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe7600037a0 con 0x7fe76c107c20 2024-03-16T10:06:09.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.581+0000 7fe76a000640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:09.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.581+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe7600cf140 con 0x7fe76c107c20 2024-03-16T10:06:09.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.581+0000 7fe770e00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:09.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.582+0000 7fe770e00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 0x7fe7540a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe75802c240 tx=0x7fe75802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:09.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.585+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe76009bf40 con 0x7fe76c107c20 2024-03-16T10:06:09.614 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:06:09.287+0000 7f8ac9e70700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T10:06:09.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: pgmap v316: 129 pgs: 16 creating+peering, 88 active+clean, 25 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-16T10:06:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T10:06:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-16T10:06:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T10:06:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3204375937' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:09 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: pgmap v316: 129 pgs: 16 creating+peering, 88 active+clean, 25 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3204375937' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:09 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: pgmap v316: 129 pgs: 16 creating+peering, 88 active+clean, 25 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-16T10:06:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T10:06:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-16T10:06:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T10:06:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3204375937' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:09 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-16T10:06:09.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.830+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe76c02d1b0 con 0x7fe76c107c20 2024-03-16T10:06:09.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.831+0000 7fe76a000640 1 -- 172.21.15.39:0/66499812 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7fe76009b840 con 0x7fe76c107c20 2024-03-16T10:06:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:09.832 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:08.798907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24569,"qdb_cluster":[24569]},"id":1}]} 2024-03-16T10:06:09.832 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 msgr2=0x7fe7540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 0x7fe7540a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe75802c240 tx=0x7fe75802df90 comp rx=0 tx=0).stop 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 msgr2=0x7fe76c19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e9c0 tx=0x7fe76000ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 shutdown_connections 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe7540a2800 0x7fe7540a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe76c1097d0 0x7fe76c1a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe76c108e10 0x7fe76c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 --2- 172.21.15.39:0/66499812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76c107c20 0x7fe76c19dee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:09.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.833+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 >> 172.21.15.39:0/66499812 conn(0x7fe76c076bb0 msgr2=0x7fe76c10c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:09.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.834+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 shutdown_connections 2024-03-16T10:06:09.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:09.834+0000 7fe772c00640 1 -- 172.21.15.39:0/66499812 wait complete. 2024-03-16T10:06:09.852 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:10.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.064+0000 7fdfa1200640 1 -- 172.21.15.39:0/336439277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c102070 msgr2=0x7fdf9c104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.064+0000 7fdfa1200640 1 --2- 172.21.15.39:0/336439277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c102070 0x7fdf9c104460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009900 tx=0x7fdf8402bc50 comp rx=0 tx=0).stop 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 -- 172.21.15.39:0/336439277 shutdown_connections 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 --2- 172.21.15.39:0/336439277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1072d0 0x7fdf9c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 --2- 172.21.15.39:0/336439277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 0x7fdf9c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 --2- 172.21.15.39:0/336439277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c102070 0x7fdf9c104460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 -- 172.21.15.39:0/336439277 >> 172.21.15.39:0/336439277 conn(0x7fdf9c0fba60 msgr2=0x7fdf9c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 -- 172.21.15.39:0/336439277 shutdown_connections 2024-03-16T10:06:10.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.065+0000 7fdfa1200640 1 -- 172.21.15.39:0/336439277 wait complete. 2024-03-16T10:06:10.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.066+0000 7fdfa1200640 1 Processor -- start 2024-03-16T10:06:10.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.066+0000 7fdfa1200640 1 -- start start 2024-03-16T10:06:10.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 0x7fdf9c100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c1072d0 0x7fdf9c100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10d260 con 0x7fdf9c1049a0 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10d0e0 con 0x7fdf9c102070 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdfa1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c10d3e0 con 0x7fdf9c1072d0 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 0x7fdf9c100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57060/0 (socket says 172.21.15.39:57060) 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 -- 172.21.15.39:0/3737901053 learned_addr learned my addr 172.21.15.39:0/3737901053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9be00640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c1072d0 0x7fdf9c100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 -- 172.21.15.39:0/3737901053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c1072d0 msgr2=0x7fdf9c100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c1072d0 0x7fdf9c100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 -- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 msgr2=0x7fdf9c100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 0x7fdf9c100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 -- 172.21.15.39:0/3737901053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c100aa0 con 0x7fdf9c102070 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.067+0000 7fdf9b400640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009230 tx=0x7fdf8402c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.068+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84042020 con 0x7fdf9c102070 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.068+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf84004450 con 0x7fdf9c102070 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.068+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84003e20 con 0x7fdf9c102070 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.068+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c100d30 con 0x7fdf9c102070 2024-03-16T10:06:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.068+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c1a63e0 con 0x7fdf9c102070 2024-03-16T10:06:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.069+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0fd550 con 0x7fdf9c102070 2024-03-16T10:06:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.071+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdf84045020 con 0x7fdf9c102070 2024-03-16T10:06:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.072+0000 7fdf93e00640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 0x7fdf800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.072+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdf840f0d90 con 0x7fdf9c102070 2024-03-16T10:06:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.072+0000 7fdf9aa00640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 0x7fdf800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.073+0000 7fdf9aa00640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 0x7fdf800a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c101750 tx=0x7fdf8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.076+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf8404c020 con 0x7fdf9c102070 2024-03-16T10:06:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.283+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf9c0623e0 con 0x7fdf9c102070 2024-03-16T10:06:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.284+0000 7fdf93e00640 1 -- 172.21.15.39:0/3737901053 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7fdf840bdb90 con 0x7fdf9c102070 2024-03-16T10:06:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:10.286 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:10.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 msgr2=0x7fdf800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 0x7fdf800a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c101750 tx=0x7fdf8c005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:10.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 msgr2=0x7fdf9c1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009230 tx=0x7fdf8402c670 comp rx=0 tx=0).stop 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 shutdown_connections 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf800a2800 0x7fdf800a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf9c1072d0 0x7fdf9c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf9c1049a0 0x7fdf9c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 --2- 172.21.15.39:0/3737901053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c102070 0x7fdf9c1019e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 >> 172.21.15.39:0/3737901053 conn(0x7fdf9c0fba60 msgr2=0x7fdf9c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 shutdown_connections 2024-03-16T10:06:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.288+0000 7fdfa1200640 1 -- 172.21.15.39:0/3737901053 wait complete. 2024-03-16T10:06:10.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 -- 172.21.15.39:0/3408751011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 msgr2=0x7fcf30108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 --2- 172.21.15.39:0/3408751011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf30108930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009900 tx=0x7fcf2802bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 -- 172.21.15.39:0/3408751011 shutdown_connections 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 --2- 172.21.15.39:0/3408751011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf30108930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 --2- 172.21.15.39:0/3408751011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 0x7fcf30101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 --2- 172.21.15.39:0/3408751011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.531+0000 7fcf2f200640 1 -- 172.21.15.39:0/3408751011 >> 172.21.15.39:0/3408751011 conn(0x7fcf300fba60 msgr2=0x7fcf300fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.532+0000 7fcf2f200640 1 -- 172.21.15.39:0/3408751011 shutdown_connections 2024-03-16T10:06:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.532+0000 7fcf2f200640 1 -- 172.21.15.39:0/3408751011 wait complete. 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.532+0000 7fcf2f200640 1 Processor -- start 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 -- start start 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 0x7fcf30199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf301a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3010c1a0 con 0x7fcf301004f0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3010c020 con 0x7fcf301020a0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3010c320 con 0x7fcf301016e0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42420/0 (socket says 172.21.15.39:42420) 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2de00640 1 -- 172.21.15.39:0/2501604332 learned_addr learned my addr 172.21.15.39:0/2501604332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2e800640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf301a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.533+0000 7fcf2d400640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 0x7fcf30199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 -- 172.21.15.39:0/2501604332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 msgr2=0x7fcf30199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 0x7fcf30199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 -- 172.21.15.39:0/2501604332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 msgr2=0x7fcf301a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf301a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 -- 172.21.15.39:0/2501604332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf301a1690 con 0x7fcf301004f0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2e800640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2de00640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e9c0 tx=0x7fcf2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000cd80 con 0x7fcf301004f0 2024-03-16T10:06:10.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf200035f0 con 0x7fcf301004f0 2024-03-16T10:06:10.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf301a1980 con 0x7fcf301004f0 2024-03-16T10:06:10.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.534+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20010690 con 0x7fcf301004f0 2024-03-16T10:06:10.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.535+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a1f20 con 0x7fcf301004f0 2024-03-16T10:06:10.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.536+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf301a21b0 con 0x7fcf301004f0 2024-03-16T10:06:10.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.538+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcf200037a0 con 0x7fcf301004f0 2024-03-16T10:06:10.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.539+0000 7fcf26a00640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 0x7fcf140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:10.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.539+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcf200cf0a0 con 0x7fcf301004f0 2024-03-16T10:06:10.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.539+0000 7fcf2d400640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 0x7fcf140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:10.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.540+0000 7fcf2d400640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 0x7fcf140a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf3019af80 tx=0x7fcf1c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:10.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.543+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcf2009bf90 con 0x7fcf301004f0 2024-03-16T10:06:10.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:10 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-16T10:06:10.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:10 smithi105 ceph-mon[36559]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T10:06:10.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:10 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:10.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:10 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/66499812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:10 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3737901053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:10 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-16T10:06:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:10 smithi194 ceph-mon[36445]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T10:06:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:10 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:10 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/66499812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:10 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3737901053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.746+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf301a2380 con 0x7fcf301004f0 2024-03-16T10:06:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.751+0000 7fcf26a00640 1 -- 172.21.15.39:0/2501604332 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7fcf2009b890 con 0x7fcf301004f0 2024-03-16T10:06:10.752 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:10.752 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.753+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 msgr2=0x7fcf140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.753+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 0x7fcf140a4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf3019af80 tx=0x7fcf1c009680 comp rx=0 tx=0).stop 2024-03-16T10:06:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.753+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 msgr2=0x7fcf30199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.753+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e9c0 tx=0x7fcf2000ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 shutdown_connections 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcf140a2880 0x7fcf140a4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf301020a0 0x7fcf301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf301016e0 0x7fcf30199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 --2- 172.21.15.39:0/2501604332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf301004f0 0x7fcf30199a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 >> 172.21.15.39:0/2501604332 conn(0x7fcf300fba60 msgr2=0x7fcf300fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 shutdown_connections 2024-03-16T10:06:10.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:10.754+0000 7fcf2f200640 1 -- 172.21.15.39:0/2501604332 wait complete. 2024-03-16T10:06:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:10 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-16T10:06:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:10 smithi039 ceph-mon[31510]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T10:06:10.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:10 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:10.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:10 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/66499812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:10 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3737901053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:10.789 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:06:11.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 -- 172.21.15.39:0/3694685895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 msgr2=0x7fa42810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 --2- 172.21.15.39:0/3694685895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa42810be40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa420009ca0 tx=0x7fa42002c0f0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 -- 172.21.15.39:0/3694685895 shutdown_connections 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 --2- 172.21.15.39:0/3694685895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa42810be40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 --2- 172.21.15.39:0/3694685895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa428105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 --2- 172.21.15.39:0/3694685895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 -- 172.21.15.39:0/3694685895 >> 172.21.15.39:0/3694685895 conn(0x7fa4280fef90 msgr2=0x7fa4281013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 -- 172.21.15.39:0/3694685895 shutdown_connections 2024-03-16T10:06:11.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.016+0000 7fa42d000640 1 -- 172.21.15.39:0/3694685895 wait complete. 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.017+0000 7fa42d000640 1 Processor -- start 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.017+0000 7fa42d000640 1 -- start start 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa42819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa4281a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810f680 con 0x7fa428104bf0 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810f500 con 0x7fa4281055b0 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa42d000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa42810f800 con 0x7fa428103a00 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa425400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa42819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa425e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa425e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:52708/0 (socket says 172.21.15.39:52708) 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa425e00640 1 -- 172.21.15.39:0/3794259743 learned_addr learned my addr 172.21.15.39:0/3794259743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.018+0000 7fa426800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa4281a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 -- 172.21.15.39:0/3794259743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 msgr2=0x7fa428199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 -- 172.21.15.39:0/3794259743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 msgr2=0x7fa4281a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa4281a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 -- 172.21.15.39:0/3794259743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281a1890 con 0x7fa428104bf0 2024-03-16T10:06:11.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa426800640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa4281a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa425400640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa42819a1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9f0 tx=0x7fa41400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400ccc0 con 0x7fa428104bf0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa414010910 con 0x7fa428104bf0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281a1b80 con 0x7fa428104bf0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414018960 con 0x7fa428104bf0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.019+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a2120 con 0x7fa428104bf0 2024-03-16T10:06:11.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.020+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42810ad20 con 0x7fa428104bf0 2024-03-16T10:06:11.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.023+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa414010ac0 con 0x7fa428104bf0 2024-03-16T10:06:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.025+0000 7fa41ea00640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 0x7fa40c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.025+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa4140cf0a0 con 0x7fa428104bf0 2024-03-16T10:06:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.025+0000 7fa425e00640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 0x7fa40c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.026+0000 7fa425e00640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 0x7fa40c0a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa42819b150 tx=0x7fa418005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.029+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa41409b850 con 0x7fa428104bf0 2024-03-16T10:06:11.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.278+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa42802d1b0 con 0x7fa428104bf0 2024-03-16T10:06:11.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.279+0000 7fa41ea00640 1 -- 172.21.15.39:0/3794259743 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7fa41409f8f0 con 0x7fa428104bf0 2024-03-16T10:06:11.279 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:11.280 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:08.798907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24569,"qdb_cluster":[24569]},"id":1}]} 2024-03-16T10:06:11.280 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 msgr2=0x7fa40c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 0x7fa40c0a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa42819b150 tx=0x7fa418005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 msgr2=0x7fa42819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa42819a1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9f0 tx=0x7fa41400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 shutdown_connections 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa40c0a2880 0x7fa40c0a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4281055b0 0x7fa4281a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa428104bf0 0x7fa42819a1c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 --2- 172.21.15.39:0/3794259743 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa428103a00 0x7fa428199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 >> 172.21.15.39:0/3794259743 conn(0x7fa4280fef90 msgr2=0x7fa428107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.282+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 shutdown_connections 2024-03-16T10:06:11.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.283+0000 7fa42d000640 1 -- 172.21.15.39:0/3794259743 wait complete. 2024-03-16T10:06:11.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 -- 172.21.15.39:0/296764413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc107c20 msgr2=0x7efefc108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 --2- 172.21.15.39:0/296764413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc107c20 0x7efefc108040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efee4009900 tx=0x7efee402bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 -- 172.21.15.39:0/296764413 shutdown_connections 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 --2- 172.21.15.39:0/296764413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc1097d0 0x7efefc110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 --2- 172.21.15.39:0/296764413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc108e10 0x7efefc109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 --2- 172.21.15.39:0/296764413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc107c20 0x7efefc108040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 -- 172.21.15.39:0/296764413 >> 172.21.15.39:0/296764413 conn(0x7efefc076bb0 msgr2=0x7efefc078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 -- 172.21.15.39:0/296764413 shutdown_connections 2024-03-16T10:06:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.521+0000 7eff00c00640 1 -- 172.21.15.39:0/296764413 wait complete. 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.522+0000 7eff00c00640 1 Processor -- start 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.522+0000 7eff00c00640 1 -- start start 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 0x7efefc1a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efefc1138e0 con 0x7efefc107c20 2024-03-16T10:06:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efefc113760 con 0x7efefc108e10 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7eff00c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efefc113a60 con 0x7efefc1097d0 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42462/0 (socket says 172.21.15.39:42462) 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57088/0 (socket says 172.21.15.39:57088) 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 -- 172.21.15.39:0/1744654060 learned_addr learned my addr 172.21.15.39:0/1744654060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefbe00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 0x7efefc1a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 -- 172.21.15.39:0/1744654060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 msgr2=0x7efefc1a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 0x7efefc1a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 -- 172.21.15.39:0/1744654060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 msgr2=0x7efefc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefb400640 1 -- 172.21.15.39:0/1744654060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc1a5b10 con 0x7efefc107c20 2024-03-16T10:06:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefaa00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.523+0000 7efefbe00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 0x7efefc1a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7efefb400640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4043020 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee40054e0 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4003ce0 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc1a5da0 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.524+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc1a61b0 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.525+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc10ef40 con 0x7efefc107c20 2024-03-16T10:06:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.527+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efee4046020 con 0x7efefc107c20 2024-03-16T10:06:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.529+0000 7efef3e00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 0x7efee00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.529+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efee40f0e20 con 0x7efefc107c20 2024-03-16T10:06:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.529+0000 7efefaa00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 0x7efee00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.530+0000 7efefaa00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 0x7efee00a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efefc19f400 tx=0x7efeec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.533+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efee404d020 con 0x7efefc107c20 2024-03-16T10:06:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:11 smithi105 ceph-mon[36559]: pgmap v319: 129 pgs: 7 creating+peering, 107 active+clean, 15 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-16T10:06:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:11 smithi105 ceph-mon[36559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-16T10:06:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:11 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:06:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:11 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2501604332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:11 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3794259743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:11 smithi194 ceph-mon[36445]: pgmap v319: 129 pgs: 7 creating+peering, 107 active+clean, 15 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-16T10:06:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:11 smithi194 ceph-mon[36445]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-16T10:06:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:11 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:06:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:11 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2501604332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:11 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3794259743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.735+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efefc062520 con 0x7efefc107c20 2024-03-16T10:06:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.737+0000 7efef3e00640 1 -- 172.21.15.39:0/1744654060 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7efee40bdc20 con 0x7efefc107c20 2024-03-16T10:06:11.738 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:11.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.740+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 msgr2=0x7efee00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.740+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 0x7efee00a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efefc19f400 tx=0x7efeec009500 comp rx=0 tx=0).stop 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.740+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 msgr2=0x7efefc19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.740+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efee4000c00 tx=0x7efee402c6c0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 shutdown_connections 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efee00a2880 0x7efee00a4d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efefc1097d0 0x7efefc1a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc108e10 0x7efefc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 --2- 172.21.15.39:0/1744654060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efefc107c20 0x7efefc19def0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 >> 172.21.15.39:0/1744654060 conn(0x7efefc076bb0 msgr2=0x7efefc10c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 shutdown_connections 2024-03-16T10:06:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.741+0000 7eff00c00640 1 -- 172.21.15.39:0/1744654060 wait complete. 2024-03-16T10:06:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:11 smithi039 ceph-mon[31510]: pgmap v319: 129 pgs: 7 creating+peering, 107 active+clean, 15 unknown; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-03-16T10:06:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:11 smithi039 ceph-mon[31510]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-16T10:06:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:11 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:06:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:11 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2501604332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:11 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3794259743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:11.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T10:06:03.734324+0000', 'modified': '2024-03-16T10:06:08.798907+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6839/2189492991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 2189492991}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 2189492991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24569, 'qdb_cluster': [24569]} 2024-03-16T10:06:11.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T10:06:11.761 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T10:06:11.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 -- 172.21.15.39:0/2494040985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1028d0 msgr2=0x7fd32c102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 --2- 172.21.15.39:0/2494040985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1028d0 0x7fd32c102d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd314009900 tx=0x7fd31402bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 -- 172.21.15.39:0/2494040985 shutdown_connections 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 --2- 172.21.15.39:0/2494040985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 --2- 172.21.15.39:0/2494040985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1028d0 0x7fd32c102d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 --2- 172.21.15.39:0/2494040985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1016e0 0x7fd32c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 -- 172.21.15.39:0/2494040985 >> 172.21.15.39:0/2494040985 conn(0x7fd32c0fcc50 msgr2=0x7fd32c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 -- 172.21.15.39:0/2494040985 shutdown_connections 2024-03-16T10:06:11.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.981+0000 7fd331000640 1 -- 172.21.15.39:0/2494040985 wait complete. 2024-03-16T10:06:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.982+0000 7fd331000640 1 Processor -- start 2024-03-16T10:06:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.982+0000 7fd331000640 1 -- start start 2024-03-16T10:06:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 0x7fd32c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d390 con 0x7fd32c1028d0 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d210 con 0x7fd32c1016e0 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd331000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c10d510 con 0x7fd32c103290 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd322a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd322a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42470/0 (socket says 172.21.15.39:42470) 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd322a00640 1 -- 172.21.15.39:0/3135473288 learned_addr learned my addr 172.21.15.39:0/3135473288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 0x7fd32c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32be00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 -- 172.21.15.39:0/3135473288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 msgr2=0x7fd32c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 -- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 msgr2=0x7fd32c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32b400640 1 -- 172.21.15.39:0/3135473288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c1a1870 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd322a00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.983+0000 7fd32be00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd32b400640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 0x7fd32c199c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd32c102730 tx=0x7fd31c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd328c00640 1 -- 172.21.15.39:0/3135473288 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c017070 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd328c00640 1 -- 172.21.15.39:0/3135473288 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c009c30 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c1a1b60 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd328c00640 1 -- 172.21.15.39:0/3135473288 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00f620 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.984+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1a2100 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 Processor -- start 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd328c00640 1 -- 172.21.15.39:0/3135473288 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd31c01a070 con 0x7fd32c1016e0 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 -- start start 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf885f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf8864ea0 0x563bf8867330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bf87c3c50 con 0x563bf885f340 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bf876c9d0 con 0x563bf885fd30 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.987+0000 7fd335439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563bf87bfcf0 con 0x563bf8864ea0 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf885f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57116/0 (socket says 172.21.15.39:57116) 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf885f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42486/0 (socket says 172.21.15.39:42486) 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 -- 172.21.15.39:0/3699870355 learned_addr learned my addr 172.21.15.39:0/3699870355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd322a00640 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf8864ea0 0x563bf8867330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 -- 172.21.15.39:0/3699870355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf8864ea0 msgr2=0x563bf8867330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf8864ea0 0x563bf8867330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 -- 172.21.15.39:0/3699870355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 msgr2=0x563bf885f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf885f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 -- 172.21.15.39:0/3699870355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf8786f10 con 0x563bf885fd30 2024-03-16T10:06:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd32b400640 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0d4c20 tx=0x7fd31c0d6670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345cdeb628d03b9e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.988+0000 7fd320c00640 1 -- 172.21.15.39:0/3699870355 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0dd070 con 0x563bf885fd30 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd320c00640 1 -- 172.21.15.39:0/3699870355 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c0cece0 con 0x563bf885fd30 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd320c00640 1 -- 172.21.15.39:0/3699870355 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c0d9790 con 0x563bf885fd30 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd328c00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 0x7fd3100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd328c00640 1 -- 172.21.15.39:0/3135473288 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd31c0ce6f0 con 0x7fd32c1016e0 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 -- 172.21.15.39:0/3699870355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 msgr2=0x563bf8864830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0d4c20 tx=0x7fd31c0d6670 comp rx=0 tx=0).stop 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd32be00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 0x7fd3100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 -- 172.21.15.39:0/3699870355 shutdown_connections 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf8864ea0 0x563bf8867330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf8864830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 --2- 172.21.15.39:0/3699870355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf885f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 -- 172.21.15.39:0/3699870355 >> 172.21.15.39:0/3699870355 conn(0x563bf885a8b0 msgr2=0x563bf885acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 -- 172.21.15.39:0/3699870355 shutdown_connections 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.989+0000 7fd335439740 1 -- 172.21.15.39:0/3699870355 wait complete. 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.990+0000 7fd32be00640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 0x7fd3100a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563bf8860590 tx=0x7fd3240076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.990+0000 7fd335439740 1 -- start start 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.990+0000 7fd335439740 1 Processor -- start 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.990+0000 7fd335439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf8870a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bf8867e50 con 0x563bf8864ea0 2024-03-16T10:06:11.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bf877fe20 con 0x563bf885f340 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563bf8867fd0 con 0x563bf885fd30 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd335439740 10 client.0 _ll_register_callbacks cb 0x563bf877f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf8870a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57132/0 (socket says 172.21.15.39:57132) 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 -- 172.21.15.39:0/2334660806 learned_addr learned my addr 172.21.15.39:0/2334660806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd322a00640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 -- 172.21.15.39:0/2334660806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 msgr2=0x563bf88699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 -- 172.21.15.39:0/2334660806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 msgr2=0x563bf8870a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf8870a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 -- 172.21.15.39:0/2334660806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf84d82d0 con 0x563bf885f340 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd32b400640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd31c097730 tx=0x7fd31c098e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229748d86a023d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:11.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.991+0000 7fd319600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c09e060 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31c0cece0 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd31c0973d0 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd31c0dd070 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 10 client.24677 fetch_fsmap learned FSMap version 17 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x563bf85a26b0 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31c0cece0 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd31c085070 con 0x563bf885f340 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 10 client.24677 fetch_fsmap finished waiting for FSMap version 17 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 10 client.24677 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 20 client.24677 populate_metadata read hostname 'smithi039' 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf8874f00 0x563bf8877060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 4 client.24677 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:06:12.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.992+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bf88212b0 con 0x563bf8874f00 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.993+0000 7fd322a00640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf8874f00 0x563bf8877060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.993+0000 7fd322a00640 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf8874f00 0x563bf8877060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c06f4c6e961097 server_cookie=f2d2ebb71f9a931c in_seq=0 out_seq=0 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.993+0000 7fd319600640 10 client.24677 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd319600640 1 -- 172.21.15.39:0/2334660806 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563bf88212b0 con 0x563bf8874f00 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd319600640 10 client.24677 handle_command_reply: tid=0 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 client.24677 shutdown 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 msgr2=0x563bf8869490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd31c097730 tx=0x7fd31c098e00 comp rx=0 tx=0).stop 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 shutdown_connections 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf8874f00 0x563bf8877060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf8870a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 --2- 172.21.15.39:0/2334660806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885f340 0x563bf8869490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 >> 172.21.15.39:0/2334660806 conn(0x563bf885a8b0 msgr2=0x563bf8859a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:12.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd319600640 0 client.24677 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:06:12.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 shutdown_connections 2024-03-16T10:06:12.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 1 -- 172.21.15.39:0/2334660806 wait complete. 2024-03-16T10:06:12.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:11.994+0000 7fd335439740 20 client.24677 trim_cache size 0 max 16384 2024-03-16T10:06:12.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 Processor -- start 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 -- start start 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf88690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bf88753a0 con 0x563bf885f340 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bf8875a70 con 0x563bf8864ea0 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.014+0000 7fd335439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563bf8876140 con 0x563bf885fd30 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57142/0 (socket says 172.21.15.39:57142) 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf88690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 -- 172.21.15.39:0/2672562113 learned_addr learned my addr 172.21.15.39:0/2672562113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd322a00640 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 -- 172.21.15.39:0/2672562113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 msgr2=0x563bf88709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 -- 172.21.15.39:0/2672562113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 msgr2=0x563bf88690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf88690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 -- 172.21.15.39:0/2672562113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf885b300 con 0x563bf8864ea0 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd322a00640 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.015+0000 7fd32be00640 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd324003c20 tx=0x7fd324014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137bb37cee9b2ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd31be00640 1 -- 172.21.15.39:0/2672562113 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32401e800 con 0x563bf8864ea0 2024-03-16T10:06:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd31be00640 1 -- 172.21.15.39:0/2672562113 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324003e50 con 0x563bf8864ea0 2024-03-16T10:06:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd31be00640 1 -- 172.21.15.39:0/2672562113 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324028e00 con 0x563bf8864ea0 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 -- 172.21.15.39:0/2672562113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 msgr2=0x563bf88695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd324003c20 tx=0x7fd324014000 comp rx=0 tx=0).stop 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 -- 172.21.15.39:0/2672562113 shutdown_connections 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf8864ea0 0x563bf88695e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885fd30 0x563bf88709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 --2- 172.21.15.39:0/2672562113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf885f340 0x563bf88690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 -- 172.21.15.39:0/2672562113 >> 172.21.15.39:0/2672562113 conn(0x563bf885a8b0 msgr2=0x563bf8855900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 -- 172.21.15.39:0/2672562113 shutdown_connections 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.016+0000 7fd335439740 1 -- 172.21.15.39:0/2672562113 wait complete. 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 -- start start 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 Processor -- start 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885f340 0x563bf886c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf886f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bf8876ee0 con 0x563bf8864ea0 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563bf8876810 con 0x563bf885fd30 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563bf88775b0 con 0x563bf885f340 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.017+0000 7fd335439740 10 client.0 _ll_register_callbacks cb 0x7fd31c0d4a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57148/0 (socket says 172.21.15.39:57148) 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd32b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885f340 0x563bf886c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 -- 172.21.15.39:0/4082644070 learned_addr learned my addr 172.21.15.39:0/4082644070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd32be00640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf886f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 -- 172.21.15.39:0/4082644070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885f340 msgr2=0x563bf886c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885f340 0x563bf886c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 -- 172.21.15.39:0/4082644070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 msgr2=0x563bf886f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf886f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 -- 172.21.15.39:0/4082644070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf886fe80 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd322a00640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd31402cec0 tx=0x7fd314000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65286ea4ee7756f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd319600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.018+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314004ad0 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd324003e50 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd314004c70 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd31405a050 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 10 client.24683 fetch_fsmap learned FSMap version 17 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x563bf88dbcb0 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd324003e50 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd31404a3e0 con 0x563bf885fd30 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 10 client.24683 fetch_fsmap finished waiting for FSMap version 17 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 10 client.24683 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 20 client.24683 populate_metadata read hostname 'smithi039' 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf88e0840 0x563bf88df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 4 client.24683 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.019+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "status"}) -- 0x563bf87d1190 con 0x563bf88e0840 2024-03-16T10:06:12.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd32b400640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf88e0840 0x563bf88df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd32b400640 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf88e0840 0x563bf88df550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=377244b580317923 server_cookie=76669b07baffaa02 in_seq=0 out_seq=0 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd319600640 10 client.24683 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd319600640 1 -- 172.21.15.39:0/4082644070 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563bf87d1190 con 0x563bf88e0840 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd319600640 10 client.24683 handle_command_reply: tid=0 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 client.24683 shutdown 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 msgr2=0x563bf886c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd31402cec0 tx=0x7fd314000bc0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 shutdown_connections 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563bf88e0840 0x563bf88df550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bf8864ea0 0x563bf886f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563bf885fd30 0x563bf886c8c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 --2- 172.21.15.39:0/4082644070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563bf885f340 0x563bf886c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd319600640 0 client.24683 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.020+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 >> 172.21.15.39:0/4082644070 conn(0x563bf885a8b0 msgr2=0x563bf88690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 shutdown_connections 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd335439740 1 -- 172.21.15.39:0/4082644070 wait complete. 2024-03-16T10:06:12.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd335439740 20 client.24683 trim_cache size 0 max 16384 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24569, 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.7897875140000004, 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 17, 2024-03-16T10:06:12.026 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 90, 2024-03-16T10:06:12.027 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 89, 2024-03-16T10:06:12.027 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 12.975290844 2024-03-16T10:06:12.027 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:06:12.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 msgr2=0x7fd3100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:12.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 0x7fd3100a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x563bf8860590 tx=0x7fd3240076a0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 msgr2=0x7fd32c199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 0x7fd32c199c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd32c102730 tx=0x7fd31c00dde0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 shutdown_connections 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3100a2880 0x7fd3100a4d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd32c103290 0x7fd32c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd32c1028d0 0x7fd32c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 --2- 172.21.15.39:0/3135473288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd32c1016e0 0x7fd32c199c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 >> 172.21.15.39:0/3135473288 conn(0x7fd32c0fcc50 msgr2=0x7fd32c073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.021+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 shutdown_connections 2024-03-16T10:06:12.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:12.022+0000 7fd331000640 1 -- 172.21.15.39:0/3135473288 wait complete. 2024-03-16T10:06:12.038 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T10:06:12.038 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:12.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T10:06:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:06:12.639 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:12 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1744654060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:12.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:12 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1744654060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:12.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:12 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1744654060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:13.675 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:13 smithi039 ceph-mon[31510]: pgmap v320: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-03-16T10:06:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:13 smithi194 ceph-mon[36445]: pgmap v320: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-03-16T10:06:13.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:13 smithi105 ceph-mon[36559]: pgmap v320: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s wr, 6 op/s 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 -- 172.21.15.39:0/2485844038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c1008c0 msgr2=0x7f532c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 --2- 172.21.15.39:0/2485844038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c1008c0 0x7f532c100d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5314009990 tx=0x7f5314009ca0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 -- 172.21.15.39:0/2485844038 shutdown_connections 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 --2- 172.21.15.39:0/2485844038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c101280 0x7f532c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 --2- 172.21.15.39:0/2485844038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c107340 0x7f532c107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 --2- 172.21.15.39:0/2485844038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c1008c0 0x7f532c100d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 -- 172.21.15.39:0/2485844038 >> 172.21.15.39:0/2485844038 conn(0x7f532c0777d0 msgr2=0x7f532c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:14.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 -- 172.21.15.39:0/2485844038 shutdown_connections 2024-03-16T10:06:14.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.811+0000 7f5331000700 1 -- 172.21.15.39:0/2485844038 wait complete. 2024-03-16T10:06:14.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.813+0000 7f5331000700 1 Processor -- start 2024-03-16T10:06:14.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.813+0000 7f5331000700 1 -- start start 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.813+0000 7f5331000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.813+0000 7f5331000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 0x7f532c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.813+0000 7f5331000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c107340 0x7f532c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5331000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f532c112770 con 0x7f532c107340 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5331000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1125f0 con 0x7f532c101280 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5331000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1128f0 con 0x7f532c1008c0 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5323e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 0x7f532c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:14.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:14.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:52810/0 (socket says 172.21.15.39:52810) 2024-03-16T10:06:14.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5323e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 0x7f532c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:57168/0 (socket says 172.21.15.39:57168) 2024-03-16T10:06:14.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 -- 172.21.15.39:0/4183156161 learned_addr learned my addr 172.21.15.39:0/4183156161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f532a800700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c107340 0x7f532c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 -- 172.21.15.39:0/4183156161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 msgr2=0x7f532c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 0x7f532c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 -- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c107340 msgr2=0x7f532c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c107340 0x7f532c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.814+0000 7f5329e00700 1 -- 172.21.15.39:0/4183156161 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c1991b0 con 0x7f532c1008c0 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5329e00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f531c00e9c0 tx=0x7f531c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:14.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c00cbd0 con 0x7f532c1008c0 2024-03-16T10:06:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c00cd70 con 0x7f532c1008c0 2024-03-16T10:06:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c1994a0 con 0x7f532c1008c0 2024-03-16T10:06:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c010430 con 0x7f532c1008c0 2024-03-16T10:06:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.815+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c1a5710 con 0x7f532c1008c0 2024-03-16T10:06:14.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.816+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532c103be0 con 0x7f532c1008c0 2024-03-16T10:06:14.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.818+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f531c004750 con 0x7f532c1008c0 2024-03-16T10:06:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.819+0000 7f5322a00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 0x7f530c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.820+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f531c0ceea0 con 0x7f532c1008c0 2024-03-16T10:06:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.820+0000 7f5323e00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 0x7f530c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:14.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.820+0000 7f5323e00700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 0x7f530c0a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f532c0ffea0 tx=0x7f5314005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:14.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:14.824+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f531c09bca0 con 0x7f532c1008c0 2024-03-16T10:06:15.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.100+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f532c199f30 con 0x7f532c1008c0 2024-03-16T10:06:15.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.101+0000 7f5322a00700 1 -- 172.21.15.39:0/4183156161 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f531c09b5a0 con 0x7f532c1008c0 2024-03-16T10:06:15.101 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-16T10:06:15.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 msgr2=0x7f530c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:15.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 0x7f530c0a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f532c0ffea0 tx=0x7f5314005fd0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 msgr2=0x7f532c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f531c00e9c0 tx=0x7f531c00ecd0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 shutdown_connections 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f530c0a2910 0x7f530c0a4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f532c107340 0x7f532c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f532c1008c0 0x7f532c19e840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 --2- 172.21.15.39:0/4183156161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f532c101280 0x7f532c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 >> 172.21.15.39:0/4183156161 conn(0x7f532c0777d0 msgr2=0x7f532c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 shutdown_connections 2024-03-16T10:06:15.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:15.104+0000 7f5331000700 1 -- 172.21.15.39:0/4183156161 wait complete. 2024-03-16T10:06:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:15 smithi194 ceph-mon[36445]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 296 B/s wr, 1 op/s 2024-03-16T10:06:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:15 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4183156161' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:06:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:15 smithi039 ceph-mon[31510]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 296 B/s wr, 1 op/s 2024-03-16T10:06:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:15 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4183156161' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:06:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:15 smithi105 ceph-mon[36559]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 296 B/s wr, 1 op/s 2024-03-16T10:06:15.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:15 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4183156161' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:06:16.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:06:16.024 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-16T10:06:16.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:16.024 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-03-16T10:06:16.082 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:06:16.082 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-03-16T10:06:16.083 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41993sec preferred_lft 41993sec 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:06:16.084 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-03-16T10:06:16.085 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T10:06:16.085 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T10:06:16.086 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:06:16.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:06:16.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:16.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:16.566 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:06:16.637 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-03-16T10:06:16.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:16.641 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-03-16T10:06:16.698 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-03-16T10:06:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:06:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T10:06:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T10:06:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:06:16.699 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:06:16.699 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:06:16.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:06:16.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:06:16.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:06:16.700 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:06:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:06:17.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:17.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:17.472 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-03-16T10:06:17.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:17.531 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-03-16T10:06:17.587 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:06:17.587 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:06:17.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:17.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-16T10:06:17.587 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:06:17.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:06:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:17 smithi194 ceph-mon[36445]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:17.735 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:17 smithi039 ceph-mon[31510]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:17.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:17.807 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-16T10:06:17.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-16T10:06:17.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-16T10:06:17.808 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:06:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:17 smithi105 ceph-mon[36559]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:06:18.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:18.163 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:06:18.163 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:06:18.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-03-16T10:06:18.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T10:06:18.163 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:06:18.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:06:18.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:18.403 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-16T10:06:18.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:06:18.403 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:18.449 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-16T10:06:18.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:06:18.450 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:18.547 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-03-16T10:06:18.647 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-03-16T10:06:18.706 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-03-16T10:06:18.707 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/896bdead6b7580dbeea0e073eb5e7b4205f1b83db909f2e6b2fac1531796795b/merged 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/ac7aab3da74cbcc120859b5b874a7e394782b4f057e826c39a6208ad35df6fc0/merged 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7242d71e8ad639ff033dcab57a2076efba62e257e6711291a4ffa7945256548b/merged 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/08a9bcce5c0a74871c796ef6f3d0feecbbcd633ac3150720714ddc0b8b9927fe/merged 2024-03-16T10:06:18.708 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/12330200c8221f048808d90169b431eab2c4ccdcc7b7c461ef53eb2abeaaa41a/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c44fc7d98379a6603bd5fcff821eb34fbbadcaac3f33bd0d96210024da7e5437/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/02a8dd54373c84fde2943e32c6fde4ef7af8738f0506ed75da2cf204ac610390/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4647a6fd1451f668f4d080ca7ea2aa40a2d5c54a34c705088b8fb1c4a760b024/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/9b15e3c22d8d433a71588c5b5c4ab1200de27f4658fc990ebdc5dcee7b29950d/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c1cec3b8f40aecc8d91f9db9b77cfb0840772e1f1556e1ac94e4d7af47faf31c/merged 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:18.709 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:18.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:18.710 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-03-16T10:06:18.767 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T10:06:18.767 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-16T10:06:18.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-03-16T10:06:18.860 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-03-16T10:06:18.906 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:06:18.907 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/896bdead6b7580dbeea0e073eb5e7b4205f1b83db909f2e6b2fac1531796795b/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/ac7aab3da74cbcc120859b5b874a7e394782b4f057e826c39a6208ad35df6fc0/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7242d71e8ad639ff033dcab57a2076efba62e257e6711291a4ffa7945256548b/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/08a9bcce5c0a74871c796ef6f3d0feecbbcd633ac3150720714ddc0b8b9927fe/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/12330200c8221f048808d90169b431eab2c4ccdcc7b7c461ef53eb2abeaaa41a/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c44fc7d98379a6603bd5fcff821eb34fbbadcaac3f33bd0d96210024da7e5437/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/02a8dd54373c84fde2943e32c6fde4ef7af8738f0506ed75da2cf204ac610390/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4647a6fd1451f668f4d080ca7ea2aa40a2d5c54a34c705088b8fb1c4a760b024/merged 2024-03-16T10:06:18.908 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/9b15e3c22d8d433a71588c5b5c4ab1200de27f4658fc990ebdc5dcee7b29950d/merged 2024-03-16T10:06:18.909 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c1cec3b8f40aecc8d91f9db9b77cfb0840772e1f1556e1ac94e4d7af47faf31c/merged 2024-03-16T10:06:18.909 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-03-16T10:06:18.909 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:18.909 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:18.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:18.910 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-03-16T10:06:19.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:2024-03-16T10:06:18.999+0000 7fea45810480 -1 init, newargv = 0x55902c9ec6a0 newargc=15 2024-03-16T10:06:19.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[75708]: starting ceph client 2024-03-16T10:06:19.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[75708]: starting fuse 2024-03-16T10:06:19.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:19 smithi194 ceph-mon[36445]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:19 smithi039 ceph-mon[31510]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:19 smithi105 ceph-mon[36559]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-16T10:06:20.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-03-16T10:06:20.218 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:06:20.279 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-03-16T10:06:20.279 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-03-16T10:06:20.279 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-03-16T10:06:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/896bdead6b7580dbeea0e073eb5e7b4205f1b83db909f2e6b2fac1531796795b/merged 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/ac7aab3da74cbcc120859b5b874a7e394782b4f057e826c39a6208ad35df6fc0/merged 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:06:20.281 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/7242d71e8ad639ff033dcab57a2076efba62e257e6711291a4ffa7945256548b/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/08a9bcce5c0a74871c796ef6f3d0feecbbcd633ac3150720714ddc0b8b9927fe/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/12330200c8221f048808d90169b431eab2c4ccdcc7b7c461ef53eb2abeaaa41a/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c44fc7d98379a6603bd5fcff821eb34fbbadcaac3f33bd0d96210024da7e5437/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/02a8dd54373c84fde2943e32c6fde4ef7af8738f0506ed75da2cf204ac610390/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4647a6fd1451f668f4d080ca7ea2aa40a2d5c54a34c705088b8fb1c4a760b024/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/9b15e3c22d8d433a71588c5b5c4ab1200de27f4658fc990ebdc5dcee7b29950d/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/c1cec3b8f40aecc8d91f9db9b77cfb0840772e1f1556e1ac94e4d7af47faf31c/merged 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:06:20.282 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:20.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:20.283 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-03-16T10:06:20.334 INFO:teuthology.orchestra.run.smithi039.stdout:118 2024-03-16T10:06:20.334 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo stdin-killer -- python3 -c ' 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> import glob 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> import re 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> import subprocess 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> def _find_admin_socket(client_name): 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> files = glob.glob(asok_path) 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> # Given a non-glob path, it better be there 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> if "*" not in asok_path: 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> assert(len(files) == 1) 2024-03-16T10:06:20.335 DEBUG:teuthology.orchestra.run.smithi039:> return files[0] 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> for f in files: 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> contents = proc_f.read() 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> if mountpoint in contents: 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> return f 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> print(_find_admin_socket("client.admin")) 2024-03-16T10:06:20.336 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-03-16T10:06:20.536 INFO:teuthology.orchestra.run.smithi039.stdout:/var/run/ceph/ceph-client.admin.75708.asok 2024-03-16T10:06:20.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:20.555 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75708.asok 2024-03-16T10:06:20.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:20.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75708.asok status 2024-03-16T10:06:20.805 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:06:20.805 INFO:teuthology.orchestra.run.smithi039.stdout: "metadata": { 2024-03-16T10:06:20.805 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_sha1": "3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-16T10:06:20.805 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T10:06:20.805 INFO:teuthology.orchestra.run.smithi039.stdout: "entity_id": "admin", 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": "smithi039", 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "pid": "75708", 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "root": "/" 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_count": 0, 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_pinned_count": 0, 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14598, 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "inst": { 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": { 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "client", 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "num": 14598 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:06:20.806 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 564530559 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 564530559 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "inst_str": "client.14598 192.168.0.1:0/564530559", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "addr_str": "192.168.0.1:0/564530559", 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "inode_count": 1, 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_epoch": 17, 2024-03-16T10:06:20.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch": 90, 2024-03-16T10:06:20.808 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch_barrier": 0, 2024-03-16T10:06:20.808 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklisted": false, 2024-03-16T10:06:20.808 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs" 2024-03-16T10:06:20.808 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:06:20.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:20.821 DEBUG:teuthology.orchestra.run.smithi039:> stat --file-system '--printf=%T 2024-03-16T10:06:20.821 DEBUG:teuthology.orchestra.run.smithi039:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:20.874 INFO:teuthology.orchestra.run.smithi039.stdout:fuseblk 2024-03-16T10:06:20.874 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:20.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:20.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:20.967 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-16T10:06:21.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:06:21.116 INFO:teuthology.orchestra.run.smithi039.stdout:# file: . 2024-03-16T10:06:21.116 INFO:teuthology.orchestra.run.smithi039.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-16T10:06:21.116 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:06:21.128 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:06:21.128 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:06:21.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:21.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 -- 172.21.15.39:0/1075969415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510074240 msgr2=0x7f35100746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 --2- 172.21.15.39:0/1075969415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510074240 0x7f35100746c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009860 tx=0x7f34fc02bb70 comp rx=0 tx=0).stop 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 -- 172.21.15.39:0/1075969415 shutdown_connections 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 --2- 172.21.15.39:0/1075969415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 0x7f351010b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 --2- 172.21.15.39:0/1075969415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510074240 0x7f35100746c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 --2- 172.21.15.39:0/1075969415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510075820 0x7f3510073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 -- 172.21.15.39:0/1075969415 >> 172.21.15.39:0/1075969415 conn(0x7f35100fcb70 msgr2=0x7f35100fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 -- 172.21.15.39:0/1075969415 shutdown_connections 2024-03-16T10:06:21.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.404+0000 7f3516400640 1 -- 172.21.15.39:0/1075969415 wait complete. 2024-03-16T10:06:21.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 Processor -- start 2024-03-16T10:06:21.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 -- start start 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510075820 0x7f351019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 0x7f35101a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f351010f440 con 0x7f3510075820 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f351010f2c0 con 0x7f3510074240 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.405+0000 7f3516400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f351010f5c0 con 0x7f35101094a0 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44342/0 (socket says 172.21.15.39:44342) 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 -- 172.21.15.39:0/1511835700 learned_addr learned my addr 172.21.15.39:0/1511835700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f350fe00640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510075820 0x7f351019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515a00640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 0x7f35101a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 -- 172.21.15.39:0/1511835700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 msgr2=0x7f35101a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 0x7f35101a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 -- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510075820 msgr2=0x7f351019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510075820 0x7f351019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.406+0000 7f3515000640 1 -- 172.21.15.39:0/1511835700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101a16f0 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f3515000640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9c0 tx=0x7f350400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350400cde0 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35040035f0 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504010690 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101a19e0 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.407+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101a1eb0 con 0x7f3510074240 2024-03-16T10:06:21.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.408+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35101081d0 con 0x7f3510074240 2024-03-16T10:06:21.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.410+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f35040037a0 con 0x7f3510074240 2024-03-16T10:06:21.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.411+0000 7f350d600640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 0x7f34f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.411+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f35040cefb0 con 0x7f3510074240 2024-03-16T10:06:21.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.411+0000 7f350fe00640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 0x7f34f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.412+0000 7f350fe00640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 0x7f34f80a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009830 tx=0x7f34fc034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:21.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.415+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f350409bdb0 con 0x7f3510074240 2024-03-16T10:06:21.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.622+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35100623e0 con 0x7f3510074240 2024-03-16T10:06:21.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.624+0000 7f350d600640 1 -- 172.21.15.39:0/1511835700 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7f35040bb070 con 0x7f3510074240 2024-03-16T10:06:21.626 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:21.626 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.627+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 msgr2=0x7f34f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.627+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 0x7f34f80a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009830 tx=0x7f34fc034ab0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.627+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 msgr2=0x7f3510199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.627+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f350400e9c0 tx=0x7f350400ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 shutdown_connections 2024-03-16T10:06:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34f80a2880 0x7f34f80a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35101094a0 0x7f35101a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3510075820 0x7f351019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 --2- 172.21.15.39:0/1511835700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3510074240 0x7f3510199ad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 >> 172.21.15.39:0/1511835700 conn(0x7f35100fcb70 msgr2=0x7f35101053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 shutdown_connections 2024-03-16T10:06:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.628+0000 7f3516400640 1 -- 172.21.15.39:0/1511835700 wait complete. 2024-03-16T10:06:21.645 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:21 smithi039 ceph-mon[31510]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:21.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:06:21.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:06:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:21 smithi194 ceph-mon[36445]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:21 smithi105 ceph-mon[36559]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 -- 172.21.15.39:0/3759640191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41029e0 msgr2=0x7f44b4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 --2- 172.21.15.39:0/3759640191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41029e0 0x7f44b4102e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f449c009900 tx=0x7f449c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 -- 172.21.15.39:0/3759640191 shutdown_connections 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 --2- 172.21.15.39:0/3759640191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41033a0 0x7f44b4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 --2- 172.21.15.39:0/3759640191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41029e0 0x7f44b4102e60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 --2- 172.21.15.39:0/3759640191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 -- 172.21.15.39:0/3759640191 >> 172.21.15.39:0/3759640191 conn(0x7f44b40fcd80 msgr2=0x7f44b40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 -- 172.21.15.39:0/3759640191 shutdown_connections 2024-03-16T10:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.885+0000 7f44bb000640 1 -- 172.21.15.39:0/3759640191 wait complete. 2024-03-16T10:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 Processor -- start 2024-03-16T10:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 -- start start 2024-03-16T10:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41029e0 0x7f44b4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44b410d4a0 con 0x7f44b41017f0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44b410d320 con 0x7f44b41033a0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44bb000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44b410d620 con 0x7f44b41029e0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.886+0000 7f44b9000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44372/0 (socket says 172.21.15.39:44372) 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 -- 172.21.15.39:0/1125005340 learned_addr learned my addr 172.21.15.39:0/1125005340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44abe00640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41029e0 0x7f44b4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 -- 172.21.15.39:0/1125005340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41029e0 msgr2=0x7f44b4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41029e0 0x7f44b4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 -- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 msgr2=0x7f44b4197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 -- 172.21.15.39:0/1125005340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b419f750 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b3e00640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b9000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009fd0 tx=0x7f44ac00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.887+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac017070 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.888+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44ac003e50 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.888+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b419fa40 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.888+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0106c0 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.888+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b419ff10 con 0x7f44b41033a0 2024-03-16T10:06:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.889+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b4108bb0 con 0x7f44b41033a0 2024-03-16T10:06:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.891+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f44ac003970 con 0x7f44b41033a0 2024-03-16T10:06:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.892+0000 7f44b1600640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 0x7f44980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:21.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.892+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f44ac0cea00 con 0x7f44b41033a0 2024-03-16T10:06:21.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.892+0000 7f44b3e00640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 0x7f44980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:21.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.893+0000 7f44b3e00640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 0x7f44980a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44b4102840 tx=0x7f44a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:21.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:21.899+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f44ac09b1b0 con 0x7f44b41033a0 2024-03-16T10:06:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.101+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f44b402d3b0 con 0x7f44b41033a0 2024-03-16T10:06:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.102+0000 7f44b1600640 1 -- 172.21.15.39:0/1125005340 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f44ac09f250 con 0x7f44b41033a0 2024-03-16T10:06:22.104 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:06:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 msgr2=0x7f44980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 0x7f44980a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44b4102840 tx=0x7f44a4005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 msgr2=0x7f44b419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009fd0 tx=0x7f44ac00b260 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 shutdown_connections 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44980a2800 0x7f44980a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44b41033a0 0x7f44b419f0e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44b41029e0 0x7f44b4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 --2- 172.21.15.39:0/1125005340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44b41017f0 0x7f44b4197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 >> 172.21.15.39:0/1125005340 conn(0x7f44b40fcd80 msgr2=0x7f44b4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 shutdown_connections 2024-03-16T10:06:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.105+0000 7f44bb000640 1 -- 172.21.15.39:0/1125005340 wait complete. 2024-03-16T10:06:22.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:06:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.356+0000 7f15b3400640 1 -- 172.21.15.39:0/2312138416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 msgr2=0x7f15ac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.356+0000 7f15b3400640 1 --2- 172.21.15.39:0/2312138416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac108220 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009900 tx=0x7f15a002bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 -- 172.21.15.39:0/2312138416 shutdown_connections 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 --2- 172.21.15.39:0/2312138416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 --2- 172.21.15.39:0/2312138416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 0x7f15ac109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 --2- 172.21.15.39:0/2312138416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac108220 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 -- 172.21.15.39:0/2312138416 >> 172.21.15.39:0/2312138416 conn(0x7f15ac076da0 msgr2=0x7f15ac0791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 -- 172.21.15.39:0/2312138416 shutdown_connections 2024-03-16T10:06:22.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.357+0000 7f15b3400640 1 -- 172.21.15.39:0/2312138416 wait complete. 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 Processor -- start 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 -- start start 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 0x7f15ac19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac1a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac113aa0 con 0x7f15ac1099b0 2024-03-16T10:06:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac113920 con 0x7f15ac107e00 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b3400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac113c20 con 0x7f15ac108ff0 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b0a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44386/0 (socket says 172.21.15.39:44386) 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.358+0000 7f15b0a00640 1 -- 172.21.15.39:0/3252101582 learned_addr learned my addr 172.21.15.39:0/3252101582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b1400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac1a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15abe00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 0x7f15ac19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 -- 172.21.15.39:0/3252101582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 msgr2=0x7f15ac19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 0x7f15ac19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 -- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 msgr2=0x7f15ac1a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac1a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 -- 172.21.15.39:0/3252101582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac1a5d10 con 0x7f15ac107e00 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b1400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac1a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15b0a00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f15a0000c00 tx=0x7f15a002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0043020 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.359+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a00054e0 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.360+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac1a5fa0 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.360+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0003ce0 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.360+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a6410 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.361+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac10f1a0 con 0x7f15ac107e00 2024-03-16T10:06:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.363+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f15a0046020 con 0x7f15ac107e00 2024-03-16T10:06:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.364+0000 7f15a9600640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 0x7f15940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.364+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f15a00f0f10 con 0x7f15ac107e00 2024-03-16T10:06:22.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.364+0000 7f15abe00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 0x7f15940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.365+0000 7f15abe00640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 0x7f15940a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1598005fd0 tx=0x7f1598005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.368+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15a004d020 con 0x7f15ac107e00 2024-03-16T10:06:22.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.584+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15ac02d1b0 con 0x7f15ac107e00 2024-03-16T10:06:22.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.585+0000 7f15a9600640 1 -- 172.21.15.39:0/3252101582 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f15a00bdd10 con 0x7f15ac107e00 2024-03-16T10:06:22.586 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:06:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 msgr2=0x7f15940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 0x7f15940a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1598005fd0 tx=0x7f1598005850 comp rx=0 tx=0).stop 2024-03-16T10:06:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 msgr2=0x7f15ac19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f15a0000c00 tx=0x7f15a002c6c0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 shutdown_connections 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f15940a2880 0x7f15940a4d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15ac1099b0 0x7f15ac1a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15ac108ff0 0x7f15ac19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 --2- 172.21.15.39:0/3252101582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15ac107e00 0x7f15ac19e0f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 >> 172.21.15.39:0/3252101582 conn(0x7f15ac076da0 msgr2=0x7f15ac10c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 shutdown_connections 2024-03-16T10:06:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.588+0000 7f15b3400640 1 -- 172.21.15.39:0/3252101582 wait complete. 2024-03-16T10:06:22.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:06:22.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:22 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1511835700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:22.685 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:22 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1125005340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:22 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1511835700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:22 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1125005340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 -- 172.21.15.39:0/906157706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa01055d0 msgr2=0x7f8fa010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 --2- 172.21.15.39:0/906157706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa01055d0 0x7f8fa010be60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009ca0 tx=0x7f8f9802c100 comp rx=0 tx=0).stop 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 -- 172.21.15.39:0/906157706 shutdown_connections 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 --2- 172.21.15.39:0/906157706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa01055d0 0x7f8fa010be60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 --2- 172.21.15.39:0/906157706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa0104c10 0x7f8fa0105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 --2- 172.21.15.39:0/906157706 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0103a20 0x7f8fa0103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 -- 172.21.15.39:0/906157706 >> 172.21.15.39:0/906157706 conn(0x7f8fa00fef90 msgr2=0x7f8fa01013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 -- 172.21.15.39:0/906157706 shutdown_connections 2024-03-16T10:06:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.828+0000 7f8fa6200640 1 -- 172.21.15.39:0/906157706 wait complete. 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.829+0000 7f8fa6200640 1 Processor -- start 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.829+0000 7f8fa6200640 1 -- start start 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0104c10 0x7f8fa019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 0x7f8fa01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010f6a0 con 0x7f8fa01055d0 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010f520 con 0x7f8fa0103a20 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8fa6200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa010f820 con 0x7f8fa0104c10 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44410/0 (socket says 172.21.15.39:44410) 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9f400640 1 -- 172.21.15.39:0/1815888499 learned_addr learned my addr 172.21.15.39:0/1815888499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9ea00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0104c10 0x7f8fa019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9fe00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 0x7f8fa01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9f400640 1 -- 172.21.15.39:0/1815888499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0104c10 msgr2=0x7f8fa019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.830+0000 7f8f9f400640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0104c10 0x7f8fa019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f9f400640 1 -- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 msgr2=0x7f8fa01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f9f400640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 0x7f8fa01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f9f400640 1 -- 172.21.15.39:0/1815888499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a1920 con 0x7f8fa0103a20 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f9fe00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 0x7f8fa01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f9f400640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d950 tx=0x7f8f9000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90017070 con 0x7f8fa0103a20 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f90009c30 con 0x7f8fa0103a20 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000f600 con 0x7f8fa0103a20 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a1c10 con 0x7f8fa0103a20 2024-03-16T10:06:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.831+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a2160 con 0x7f8fa0103a20 2024-03-16T10:06:22.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.832+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa010ad40 con 0x7f8fa0103a20 2024-03-16T10:06:22.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.834+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8f9001a070 con 0x7f8fa0103a20 2024-03-16T10:06:22.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.835+0000 7f8f97e00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 0x7f8f840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:22.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.835+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8f900ce710 con 0x7f8fa0103a20 2024-03-16T10:06:22.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.836+0000 7f8f9ea00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 0x7f8f840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:22.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.836+0000 7f8f9ea00640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 0x7f8f840a4d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009bd0 tx=0x7f8f8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:22.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:22.839+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8f90009de0 con 0x7f8fa0103a20 2024-03-16T10:06:22.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:22 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1511835700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:22.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:22 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1125005340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.044+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8fa002d1b0 con 0x7f8fa0103a20 2024-03-16T10:06:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.045+0000 7f8f97e00640 1 -- 172.21.15.39:0/1815888499 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f8fa002d1b0 con 0x7f8fa0103a20 2024-03-16T10:06:23.049 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.047+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 msgr2=0x7f8f840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.047+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 0x7f8f840a4d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009bd0 tx=0x7f8f8c005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.047+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 msgr2=0x7f8fa0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.047+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000d950 tx=0x7f8f9000de10 comp rx=0 tx=0).stop 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 shutdown_connections 2024-03-16T10:06:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8f840a2880 0x7f8f840a4d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fa01055d0 0x7f8fa01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8fa0104c10 0x7f8fa019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 --2- 172.21.15.39:0/1815888499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fa0103a20 0x7f8fa0199d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 >> 172.21.15.39:0/1815888499 conn(0x7f8fa00fef90 msgr2=0x7f8fa0107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 shutdown_connections 2024-03-16T10:06:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.048+0000 7f8fa6200640 1 -- 172.21.15.39:0/1815888499 wait complete. 2024-03-16T10:06:23.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:06:23.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.293+0000 7faf38a00640 1 -- 172.21.15.39:0/1461656408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf34104c10 msgr2=0x7faf34105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.293+0000 7faf38a00640 1 --2- 172.21.15.39:0/1461656408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf34104c10 0x7faf34105090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009900 tx=0x7faf1c02bc50 comp rx=0 tx=0).stop 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 -- 172.21.15.39:0/1461656408 shutdown_connections 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 --2- 172.21.15.39:0/1461656408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf341055d0 0x7faf3410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 --2- 172.21.15.39:0/1461656408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf34104c10 0x7faf34105090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 --2- 172.21.15.39:0/1461656408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 -- 172.21.15.39:0/1461656408 >> 172.21.15.39:0/1461656408 conn(0x7faf340fef90 msgr2=0x7faf341013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 -- 172.21.15.39:0/1461656408 shutdown_connections 2024-03-16T10:06:23.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.294+0000 7faf38a00640 1 -- 172.21.15.39:0/1461656408 wait complete. 2024-03-16T10:06:23.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.295+0000 7faf38a00640 1 Processor -- start 2024-03-16T10:06:23.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.295+0000 7faf38a00640 1 -- start start 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 0x7faf3419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf341055d0 0x7faf341a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410f6d0 con 0x7faf34104c10 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410f550 con 0x7faf34103a20 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf38a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf3410f850 con 0x7faf341055d0 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf31e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf31e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44418/0 (socket says 172.21.15.39:44418) 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf31e00640 1 -- 172.21.15.39:0/1800576942 learned_addr learned my addr 172.21.15.39:0/1800576942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf31400640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 0x7faf3419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.296+0000 7faf32800640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf341055d0 0x7faf341a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 -- 172.21.15.39:0/1800576942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf341055d0 msgr2=0x7faf341a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf341055d0 0x7faf341a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 -- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 msgr2=0x7faf3419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 0x7faf3419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 -- 172.21.15.39:0/1800576942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a1950 con 0x7faf34103a20 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31400640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 0x7faf3419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf31e00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf2400ebc0 tx=0x7faf2400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24003b70 con 0x7faf34103a20 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf24003d10 con 0x7faf34103a20 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a1c40 con 0x7faf34103a20 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24019980 con 0x7faf34103a20 2024-03-16T10:06:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.297+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a2190 con 0x7faf34103a20 2024-03-16T10:06:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.298+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3410ad40 con 0x7faf34103a20 2024-03-16T10:06:23.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.301+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7faf24019b20 con 0x7faf34103a20 2024-03-16T10:06:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.302+0000 7faf2aa00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 0x7faf180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.302+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faf240cf610 con 0x7faf34103a20 2024-03-16T10:06:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.303+0000 7faf31400640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 0x7faf180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.304+0000 7faf31400640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 0x7faf180a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf1c02c260 tx=0x7faf1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.306+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faf2409c470 con 0x7faf34103a20 2024-03-16T10:06:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.511+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faf34062480 con 0x7faf34103a20 2024-03-16T10:06:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.512+0000 7faf2aa00640 1 -- 172.21.15.39:0/1800576942 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7faf2409bd70 con 0x7faf34103a20 2024-03-16T10:06:23.513 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:06:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.514+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 msgr2=0x7faf180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.514+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 0x7faf180a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf1c02c260 tx=0x7faf1c005fd0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.514+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 msgr2=0x7faf34199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.514+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faf2400ebc0 tx=0x7faf2400c6a0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 shutdown_connections 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faf180a2880 0x7faf180a4d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf341055d0 0x7faf341a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf34104c10 0x7faf3419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 --2- 172.21.15.39:0/1800576942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf34103a20 0x7faf34199d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 >> 172.21.15.39:0/1800576942 conn(0x7faf340fef90 msgr2=0x7faf34107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 shutdown_connections 2024-03-16T10:06:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.515+0000 7faf38a00640 1 -- 172.21.15.39:0/1800576942 wait complete. 2024-03-16T10:06:23.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:06:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:23 smithi194 ceph-mon[36445]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:23 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3252101582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:23 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1815888499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 -- 172.21.15.39:0/2544898469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 msgr2=0x7fdcec102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 --2- 172.21.15.39:0/2544898469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec102e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009ca0 tx=0x7fdcd402c0f0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 -- 172.21.15.39:0/2544898469 shutdown_connections 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 --2- 172.21.15.39:0/2544898469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1033a0 0x7fdcec109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 --2- 172.21.15.39:0/2544898469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec102e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 --2- 172.21.15.39:0/2544898469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1017f0 0x7fdcec101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 -- 172.21.15.39:0/2544898469 >> 172.21.15.39:0/2544898469 conn(0x7fdcec0fcd60 msgr2=0x7fdcec0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 -- 172.21.15.39:0/2544898469 shutdown_connections 2024-03-16T10:06:23.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.750+0000 7fdcf1200640 1 -- 172.21.15.39:0/2544898469 wait complete. 2024-03-16T10:06:23.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.751+0000 7fdcf1200640 1 Processor -- start 2024-03-16T10:06:23.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 -- start start 2024-03-16T10:06:23.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 0x7fdcec199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1033a0 0x7fdcec1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d470 con 0x7fdcec1017f0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d2f0 con 0x7fdcec1029e0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdcf1200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10d5f0 con 0x7fdcec1033a0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 0x7fdcec199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.752+0000 7fdce9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 0x7fdcec199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36036/0 (socket says 172.21.15.39:36036) 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44426/0 (socket says 172.21.15.39:44426) 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9e00640 1 -- 172.21.15.39:0/4215831159 learned_addr learned my addr 172.21.15.39:0/4215831159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdcea800640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1033a0 0x7fdcec1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 -- 172.21.15.39:0/4215831159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1033a0 msgr2=0x7fdcec1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1033a0 0x7fdcec1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 -- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 msgr2=0x7fdcec199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 0x7fdcec199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 -- 172.21.15.39:0/4215831159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec1a19d0 con 0x7fdcec1029e0 2024-03-16T10:06:23.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:23 smithi039 ceph-mon[31510]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:23.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:23 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3252101582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:23 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1815888499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.753+0000 7fdce9400640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009bf0 tx=0x7fdcd4032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.754+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4041020 con 0x7fdcec1029e0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.754+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcd4034cd0 con 0x7fdcec1029e0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.754+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1a1c60 con 0x7fdcec1029e0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.754+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4039040 con 0x7fdcec1029e0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.754+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a2170 con 0x7fdcec1029e0 2024-03-16T10:06:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.755+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec1a2340 con 0x7fdcec1029e0 2024-03-16T10:06:23.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.757+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdcd4044020 con 0x7fdcec1029e0 2024-03-16T10:06:23.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.758+0000 7fdce2a00640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 0x7fdcd00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:23.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.758+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdcd40f0940 con 0x7fdcec1029e0 2024-03-16T10:06:23.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.758+0000 7fdce9e00640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 0x7fdcd00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:23.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.759+0000 7fdce9e00640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 0x7fdcd00a4d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcec19b290 tx=0x7fdce4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:23.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.762+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdcd404b020 con 0x7fdcec1029e0 2024-03-16T10:06:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:23 smithi105 ceph-mon[36559]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:23 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3252101582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:23 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1815888499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:23.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.979+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdcec1a2510 con 0x7fdcec1029e0 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.979+0000 7fdce2a00640 1 -- 172.21.15.39:0/4215831159 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7fdcd40bd7a0 con 0x7fdcec1029e0 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3114779354 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3549110681 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1618736720 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/865907722 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4208944392 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:23.982 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:23.982 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:23.982 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3617878948 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:23.982 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:listed 10 entries 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 msgr2=0x7fdcd00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 0x7fdcd00a4d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdcec19b290 tx=0x7fdce4005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 msgr2=0x7fdcec19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009bf0 tx=0x7fdcd4032ec0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 shutdown_connections 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdcd00a2880 0x7fdcd00a4d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1033a0 0x7fdcec1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcec1029e0 0x7fdcec19a350 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 --2- 172.21.15.39:0/4215831159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcec1017f0 0x7fdcec199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 >> 172.21.15.39:0/4215831159 conn(0x7fdcec0fcd60 msgr2=0x7fdcec073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 shutdown_connections 2024-03-16T10:06:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:23.982+0000 7fdcf1200640 1 -- 172.21.15.39:0/4215831159 wait complete. 2024-03-16T10:06:24.003 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:06:24.003 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:06:24.060 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 -- 172.21.15.39:0/3372812940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 msgr2=0x7f95d4102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 --2- 172.21.15.39:0/3372812940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4102e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009900 tx=0x7f95c002bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 -- 172.21.15.39:0/3372812940 shutdown_connections 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 --2- 172.21.15.39:0/3372812940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 --2- 172.21.15.39:0/3372812940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4102e50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 --2- 172.21.15.39:0/3372812940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 -- 172.21.15.39:0/3372812940 >> 172.21.15.39:0/3372812940 conn(0x7f95d40fcd50 msgr2=0x7f95d40ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 -- 172.21.15.39:0/3372812940 shutdown_connections 2024-03-16T10:06:24.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.278+0000 7f95dc600640 1 -- 172.21.15.39:0/3372812940 wait complete. 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 Processor -- start 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 -- start start 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95d410d490 con 0x7f95d41017e0 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95d410d310 con 0x7f95d4103390 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95dc600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95d410d610 con 0x7f95d41029d0 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.280+0000 7f95d9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95d9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36042/0 (socket says 172.21.15.39:36042) 2024-03-16T10:06:24.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95d9c00640 1 -- 172.21.15.39:0/1917864259 learned_addr learned my addr 172.21.15.39:0/1917864259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:24.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95d9200640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 -- 172.21.15.39:0/1917864259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 msgr2=0x7f95d4073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 -- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 msgr2=0x7f95d4075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 -- 172.21.15.39:0/1917864259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d40749a0 con 0x7f95d4103390 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95d9200640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95d9c00640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.281+0000 7f95da600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4074110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e970 tx=0x7f95cc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.282+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00ccc0 con 0x7f95d4103390 2024-03-16T10:06:24.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.282+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc0035f0 con 0x7f95d4103390 2024-03-16T10:06:24.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.282+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00ce60 con 0x7f95d4103390 2024-03-16T10:06:24.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.282+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d41a1c00 con 0x7f95d4103390 2024-03-16T10:06:24.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.282+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d41a20d0 con 0x7f95d4103390 2024-03-16T10:06:24.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.283+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d4068a90 con 0x7f95d4103390 2024-03-16T10:06:24.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.285+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f95cc0037a0 con 0x7f95d4103390 2024-03-16T10:06:24.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.286+0000 7f95d2000640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 0x7f95bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.286+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f95cc0cefe0 con 0x7f95d4103390 2024-03-16T10:06:24.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.287+0000 7f95d9c00640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 0x7f95bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.287+0000 7f95d9c00640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 0x7f95bc0a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95c800f7b0 tx=0x7f95c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:24.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.290+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95cc09be40 con 0x7f95d4103390 2024-03-16T10:06:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.501+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d40623e0 con 0x7f95d4103390 2024-03-16T10:06:24.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.503+0000 7f95d2000640 1 -- 172.21.15.39:0/1917864259 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7f95cc09b740 con 0x7f95d4103390 2024-03-16T10:06:24.505 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:24.505 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 msgr2=0x7f95bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 0x7f95bc0a4cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95c800f7b0 tx=0x7f95c8009500 comp rx=0 tx=0).stop 2024-03-16T10:06:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 msgr2=0x7f95d4074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4074110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00e970 tx=0x7f95cc00ee30 comp rx=0 tx=0).stop 2024-03-16T10:06:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 shutdown_connections 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f95bc0a2800 0x7f95bc0a4cc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d4103390 0x7f95d4074110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95d41029d0 0x7f95d4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 --2- 172.21.15.39:0/1917864259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95d41017e0 0x7f95d4075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 >> 172.21.15.39:0/1917864259 conn(0x7f95d40fcd50 msgr2=0x7f95d4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 shutdown_connections 2024-03-16T10:06:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.506+0000 7f95dc600640 1 -- 172.21.15.39:0/1917864259 wait complete. 2024-03-16T10:06:24.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:06:24.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:06:24.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:24 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1800576942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:24.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:24 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4215831159' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:24.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.752+0000 7fad4fa00640 1 -- 172.21.15.39:0/988475441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad480ffa30 msgr2=0x7fad480ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.752+0000 7fad4fa00640 1 --2- 172.21.15.39:0/988475441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad480ffa30 0x7fad480ffe50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0074e0 tx=0x7fad3c02c8d0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 -- 172.21.15.39:0/988475441 shutdown_connections 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 --2- 172.21.15.39:0/988475441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad481015e0 0x7fad48107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 --2- 172.21.15.39:0/988475441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad48100c20 0x7fad481010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 --2- 172.21.15.39:0/988475441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad480ffa30 0x7fad480ffe50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 -- 172.21.15.39:0/988475441 >> 172.21.15.39:0/988475441 conn(0x7fad480fafa0 msgr2=0x7fad480fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 -- 172.21.15.39:0/988475441 shutdown_connections 2024-03-16T10:06:24.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.753+0000 7fad4fa00640 1 -- 172.21.15.39:0/988475441 wait complete. 2024-03-16T10:06:24.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:24 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1800576942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:24.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:24 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4215831159' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.754+0000 7fad4fa00640 1 Processor -- start 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.754+0000 7fad4fa00640 1 -- start start 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.754+0000 7fad4fa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.754+0000 7fad4fa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad48100c20 0x7fad48198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.754+0000 7fad4fa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 0x7fad4819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4fa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810b240 con 0x7fad480ffa30 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4fa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810b0c0 con 0x7fad481015e0 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4fa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fad4810b3c0 con 0x7fad48100c20 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36056/0 (socket says 172.21.15.39:36056) 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 -- 172.21.15.39:0/597504383 learned_addr learned my addr 172.21.15.39:0/597504383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4da00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 0x7fad4819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad47e00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad48100c20 0x7fad48198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 -- 172.21.15.39:0/597504383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad48100c20 msgr2=0x7fad48198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad48100c20 0x7fad48198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 -- 172.21.15.39:0/597504383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 msgr2=0x7fad4819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 0x7fad4819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4d000640 1 -- 172.21.15.39:0/597504383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4819f710 con 0x7fad480ffa30 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.755+0000 7fad4da00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 0x7fad4819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:24.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad4d000640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad3c02cee0 tx=0x7fad3c00a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c040070 con 0x7fad480ffa30 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad3c0051d0 con 0x7fad480ffa30 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4819f9a0 con 0x7fad480ffa30 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c0046c0 con 0x7fad480ffa30 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.756+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4819fef0 con 0x7fad480ffa30 2024-03-16T10:06:24.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.757+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad48068a20 con 0x7fad480ffa30 2024-03-16T10:06:24.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.759+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fad3c039070 con 0x7fad480ffa30 2024-03-16T10:06:24.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.762+0000 7fad45600640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 0x7fad300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:24.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.762+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fad3c0efe50 con 0x7fad480ffa30 2024-03-16T10:06:24.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.762+0000 7fad47e00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 0x7fad300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:24.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.763+0000 7fad47e00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 0x7fad300a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad48199000 tx=0x7fad38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:24.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.765+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fad3c049030 con 0x7fad480ffa30 2024-03-16T10:06:24.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:24 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1800576942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:24.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:24 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4215831159' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:24.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.967+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad481a0310 con 0x7fad480ffa30 2024-03-16T10:06:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.968+0000 7fad45600640 1 -- 172.21.15.39:0/597504383 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fad3c0bccb0 con 0x7fad480ffa30 2024-03-16T10:06:24.970 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:06:24.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 msgr2=0x7fad300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 0x7fad300a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fad48199000 tx=0x7fad38005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 msgr2=0x7fad48197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad3c02cee0 tx=0x7fad3c00a380 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 shutdown_connections 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fad300a2800 0x7fad300a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad481015e0 0x7fad4819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad48100c20 0x7fad48198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 --2- 172.21.15.39:0/597504383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad480ffa30 0x7fad48197af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 >> 172.21.15.39:0/597504383 conn(0x7fad480fafa0 msgr2=0x7fad481061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 shutdown_connections 2024-03-16T10:06:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:24.971+0000 7fad4fa00640 1 -- 172.21.15.39:0/597504383 wait complete. 2024-03-16T10:06:24.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 -- 172.21.15.39:0/859531573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010b410 msgr2=0x7fcda010d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 --2- 172.21.15.39:0/859531573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010b410 0x7fcda010d800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009860 tx=0x7fcd8c02bb70 comp rx=0 tx=0).stop 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 -- 172.21.15.39:0/859531573 shutdown_connections 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 --2- 172.21.15.39:0/859531573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010de90 0x7fcda0110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 --2- 172.21.15.39:0/859531573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010b410 0x7fcda010d800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 --2- 172.21.15.39:0/859531573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 -- 172.21.15.39:0/859531573 >> 172.21.15.39:0/859531573 conn(0x7fcda006e9e0 msgr2=0x7fcda0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 -- 172.21.15.39:0/859531573 shutdown_connections 2024-03-16T10:06:25.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.215+0000 7fcda7e00640 1 -- 172.21.15.39:0/859531573 wait complete. 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.216+0000 7fcda7e00640 1 Processor -- start 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.216+0000 7fcda7e00640 1 -- start start 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010b410 0x7fcda01a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010de90 0x7fcda01a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0114b40 con 0x7fcda010b410 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01149c0 con 0x7fcda0108a50 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0114cc0 con 0x7fcda010de90 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010b410 0x7fcda01a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda5400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44480/0 (socket says 172.21.15.39:44480) 2024-03-16T10:06:25.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda5400640 1 -- 172.21.15.39:0/3430423677 learned_addr learned my addr 172.21.15.39:0/3430423677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.217+0000 7fcda5e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010de90 0x7fcda01a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 -- 172.21.15.39:0/3430423677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010de90 msgr2=0x7fcda01a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010de90 0x7fcda01a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 -- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010b410 msgr2=0x7fcda01a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010b410 0x7fcda01a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 -- 172.21.15.39:0/3430423677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda01aa160 con 0x7fcda0108a50 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda5400640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e7a0 tx=0x7fcd9400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94018030 con 0x7fcda0108a50 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9400cd30 con 0x7fcda0108a50 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94004a00 con 0x7fcda0108a50 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda01aa450 con 0x7fcda0108a50 2024-03-16T10:06:25.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.218+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda01aa970 con 0x7fcda0108a50 2024-03-16T10:06:25.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.219+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda01aac30 con 0x7fcda0108a50 2024-03-16T10:06:25.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.221+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fcd9401b070 con 0x7fcda0108a50 2024-03-16T10:06:25.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.223+0000 7fcd9e000640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.223+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcd940ce8b0 con 0x7fcda0108a50 2024-03-16T10:06:25.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.223+0000 7fcda4a00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.224+0000 7fcda4a00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 0x7fcd880a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0349e0 tx=0x7fcd8c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:25.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.227+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd94010040 con 0x7fcda0108a50 2024-03-16T10:06:25.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.432+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcda01aae00 con 0x7fcda0108a50 2024-03-16T10:06:25.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.433+0000 7fcd9e000640 1 -- 172.21.15.39:0/3430423677 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fcd9409b6b0 con 0x7fcda0108a50 2024-03-16T10:06:25.434 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:06:25.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 msgr2=0x7fcd880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 0x7fcd880a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0349e0 tx=0x7fcd8c034ab0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 msgr2=0x7fcda01a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400e7a0 tx=0x7fcd9400ec60 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 shutdown_connections 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcda010de90 0x7fcda01a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda010b410 0x7fcda01a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 --2- 172.21.15.39:0/3430423677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcda0108a50 0x7fcda01a2540 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 >> 172.21.15.39:0/3430423677 conn(0x7fcda006e9e0 msgr2=0x7fcda0111760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 shutdown_connections 2024-03-16T10:06:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.436+0000 7fcda7e00640 1 -- 172.21.15.39:0/3430423677 wait complete. 2024-03-16T10:06:25.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 -- 172.21.15.39:0/2538494479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf501038e0 msgr2=0x7fdf50103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 --2- 172.21.15.39:0/2538494479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf501038e0 0x7fdf50103d00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009900 tx=0x7fdf4002bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 -- 172.21.15.39:0/2538494479 shutdown_connections 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 --2- 172.21.15.39:0/2538494479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 0x7fdf5010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 --2- 172.21.15.39:0/2538494479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf50104ad0 0x7fdf50104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 --2- 172.21.15.39:0/2538494479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf501038e0 0x7fdf50103d00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 -- 172.21.15.39:0/2538494479 >> 172.21.15.39:0/2538494479 conn(0x7fdf500fee50 msgr2=0x7fdf50101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:25.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 -- 172.21.15.39:0/2538494479 shutdown_connections 2024-03-16T10:06:25.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.686+0000 7fdf55200640 1 -- 172.21.15.39:0/2538494479 wait complete. 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.687+0000 7fdf55200640 1 Processor -- start 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.687+0000 7fdf55200640 1 -- start start 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 0x7fdf50199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 0x7fdf501a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010f5b0 con 0x7fdf501038e0 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010f430 con 0x7fdf50104ad0 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf55200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010f730 con 0x7fdf50105490 2024-03-16T10:06:25.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44490/0 (socket says 172.21.15.39:44490) 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 -- 172.21.15.39:0/775302114 learned_addr learned my addr 172.21.15.39:0/775302114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4f400640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 0x7fdf50199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 0x7fdf501a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 -- 172.21.15.39:0/775302114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 msgr2=0x7fdf501a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 0x7fdf501a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 -- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 msgr2=0x7fdf50199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 0x7fdf50199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4ea00640 1 -- 172.21.15.39:0/775302114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf501a1810 con 0x7fdf50104ad0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.688+0000 7fdf4f400640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 0x7fdf50199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf4ea00640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00d950 tx=0x7fdf3c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c017030 con 0x7fdf50104ad0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c004020 con 0x7fdf50104ad0 2024-03-16T10:06:25.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0047a0 con 0x7fdf50104ad0 2024-03-16T10:06:25.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf501a1b00 con 0x7fdf50104ad0 2024-03-16T10:06:25.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.689+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501a2000 con 0x7fdf50104ad0 2024-03-16T10:06:25.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.690+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5010ac00 con 0x7fdf50104ad0 2024-03-16T10:06:25.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.693+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdf3c0041d0 con 0x7fdf50104ad0 2024-03-16T10:06:25.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.694+0000 7fdf47e00640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:25.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.694+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdf3c0ce6f0 con 0x7fdf50104ad0 2024-03-16T10:06:25.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.694+0000 7fdf4f400640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:25.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.695+0000 7fdf4f400640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 0x7fdf340a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf4002c170 tx=0x7fdf4002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:25.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.698+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdf3c09b4f0 con 0x7fdf50104ad0 2024-03-16T10:06:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:25 smithi194 ceph-mon[36445]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:25 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1917864259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:25 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/597504383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:25 smithi039 ceph-mon[31510]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:25 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1917864259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:25 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/597504383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:25 smithi105 ceph-mon[36559]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:25 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1917864259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:25 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/597504383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:06:25.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.902+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf50062410 con 0x7fdf50104ad0 2024-03-16T10:06:25.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.903+0000 7fdf47e00640 1 -- 172.21.15.39:0/775302114 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fdf3c09adf0 con 0x7fdf50104ad0 2024-03-16T10:06:25.904 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:06:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.905+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 msgr2=0x7fdf340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.905+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 0x7fdf340a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf4002c170 tx=0x7fdf4002df90 comp rx=0 tx=0).stop 2024-03-16T10:06:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.905+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 msgr2=0x7fdf5019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.905+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00d950 tx=0x7fdf3c00de10 comp rx=0 tx=0).stop 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 shutdown_connections 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdf340a2880 0x7fdf340a4d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf50105490 0x7fdf501a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf50104ad0 0x7fdf5019a130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 --2- 172.21.15.39:0/775302114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf501038e0 0x7fdf50199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 >> 172.21.15.39:0/775302114 conn(0x7fdf500fee50 msgr2=0x7fdf50107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 shutdown_connections 2024-03-16T10:06:25.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:25.906+0000 7fdf55200640 1 -- 172.21.15.39:0/775302114 wait complete. 2024-03-16T10:06:25.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:06:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 -- 172.21.15.39:0/2774539395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe690103900 msgr2=0x7fe690103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 --2- 172.21.15.39:0/2774539395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe690103900 0x7fe690103d20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe684009ca0 tx=0x7fe68402c100 comp rx=0 tx=0).stop 2024-03-16T10:06:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 -- 172.21.15.39:0/2774539395 shutdown_connections 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 --2- 172.21.15.39:0/2774539395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6901054b0 0x7fe69010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 --2- 172.21.15.39:0/2774539395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690104af0 0x7fe690104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 --2- 172.21.15.39:0/2774539395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe690103900 0x7fe690103d20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.155+0000 7fe696e00640 1 -- 172.21.15.39:0/2774539395 >> 172.21.15.39:0/2774539395 conn(0x7fe6900fee90 msgr2=0x7fe6901012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.156+0000 7fe696e00640 1 -- 172.21.15.39:0/2774539395 shutdown_connections 2024-03-16T10:06:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.156+0000 7fe696e00640 1 -- 172.21.15.39:0/2774539395 wait complete. 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 Processor -- start 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 -- start start 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 0x7fe69019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe69010f580 con 0x7fe690103900 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe69010f400 con 0x7fe6901054b0 2024-03-16T10:06:26.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.157+0000 7fe696e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe69010f700 con 0x7fe690104af0 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe695a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe695000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 0x7fe69019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44498/0 (socket says 172.21.15.39:44498) 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe695a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36110/0 (socket says 172.21.15.39:36110) 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 -- 172.21.15.39:0/3379866098 learned_addr learned my addr 172.21.15.39:0/3379866098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 -- 172.21.15.39:0/3379866098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 msgr2=0x7fe69019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 0x7fe69019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 -- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 msgr2=0x7fe690199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 -- 172.21.15.39:0/3379866098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6901a17f0 con 0x7fe6901054b0 2024-03-16T10:06:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe695000640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 0x7fe69019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe695a00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.158+0000 7fe696400640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe68800d750 tx=0x7fe68800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.159+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688017070 con 0x7fe6901054b0 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.159+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe688005500 con 0x7fe6901054b0 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.159+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004d90 con 0x7fe6901054b0 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.159+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6901a1ae0 con 0x7fe6901054b0 2024-03-16T10:06:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.159+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a2030 con 0x7fe6901054b0 2024-03-16T10:06:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.160+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69010ac20 con 0x7fe6901054b0 2024-03-16T10:06:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.162+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe688004f30 con 0x7fe6901054b0 2024-03-16T10:06:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.163+0000 7fe68e000640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 0x7fe6780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.163+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6880ce560 con 0x7fe6901054b0 2024-03-16T10:06:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.164+0000 7fe695a00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 0x7fe6780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.164+0000 7fe695a00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 0x7fe6780a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe68402c640 tx=0x7fe684009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:26.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.167+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe68809ad10 con 0x7fe6901054b0 2024-03-16T10:06:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.371+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe6900623e0 con 0x7fe6901054b0 2024-03-16T10:06:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.372+0000 7fe68e000640 1 -- 172.21.15.39:0/3379866098 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fe68809edb0 con 0x7fe6901054b0 2024-03-16T10:06:26.373 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:06:26.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.374+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 msgr2=0x7fe6780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.374+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 0x7fe6780a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe68402c640 tx=0x7fe684009360 comp rx=0 tx=0).stop 2024-03-16T10:06:26.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.374+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 msgr2=0x7fe6901a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.374+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe68800d750 tx=0x7fe68800dc10 comp rx=0 tx=0).stop 2024-03-16T10:06:26.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.374+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 shutdown_connections 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe6780a2880 0x7fe6780a4d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6901054b0 0x7fe6901a1180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe690104af0 0x7fe69019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 --2- 172.21.15.39:0/3379866098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe690103900 0x7fe690199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 >> 172.21.15.39:0/3379866098 conn(0x7fe6900fee90 msgr2=0x7fe690107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 shutdown_connections 2024-03-16T10:06:26.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.375+0000 7fe696e00640 1 -- 172.21.15.39:0/3379866098 wait complete. 2024-03-16T10:06:26.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:06:26.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3430423677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/775302114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:26 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3379866098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.611+0000 7f279f000640 1 -- 172.21.15.39:0/1050631865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2798107520 msgr2=0x7f2798109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.611+0000 7f279f000640 1 --2- 172.21.15.39:0/1050631865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2798107520 0x7f2798109930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2788009900 tx=0x7f278802bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:26.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 -- 172.21.15.39:0/1050631865 shutdown_connections 2024-03-16T10:06:26.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 --2- 172.21.15.39:0/1050631865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2798107520 0x7f2798109930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 --2- 172.21.15.39:0/1050631865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798104bf0 0x7f2798106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 --2- 172.21.15.39:0/1050631865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27981022c0 0x7f27981046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 -- 172.21.15.39:0/1050631865 >> 172.21.15.39:0/1050631865 conn(0x7f27980fbc90 msgr2=0x7f27980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 -- 172.21.15.39:0/1050631865 shutdown_connections 2024-03-16T10:06:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.612+0000 7f279f000640 1 -- 172.21.15.39:0/1050631865 wait complete. 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 Processor -- start 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 -- start start 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 0x7f279819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798107520 0x7f27981a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f279810d4d0 con 0x7f2798107520 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f279810d350 con 0x7f27981022c0 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279f000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f279810d650 con 0x7f2798104bf0 2024-03-16T10:06:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f2797e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f2797e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44526/0 (socket says 172.21.15.39:44526) 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f2797e00640 1 -- 172.21.15.39:0/2755692131 learned_addr learned my addr 172.21.15.39:0/2755692131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.613+0000 7f279d000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798107520 0x7f27981a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797400640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 0x7f279819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 -- 172.21.15.39:0/2755692131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 msgr2=0x7f279819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 0x7f279819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 -- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798107520 msgr2=0x7f27981a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798107520 0x7f27981a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 -- 172.21.15.39:0/2755692131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981a18f0 con 0x7f27981022c0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797400640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 0x7f279819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.614+0000 7f2797e00640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f279000eab0 tx=0x7f279000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.615+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000ce40 con 0x7f27981022c0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.615+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2790011950 con 0x7f27981022c0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.615+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981a1be0 con 0x7f27981022c0 2024-03-16T10:06:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.615+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790010ae0 con 0x7f27981022c0 2024-03-16T10:06:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.615+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981a2180 con 0x7f27981022c0 2024-03-16T10:06:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.616+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2798109010 con 0x7f27981022c0 2024-03-16T10:06:26.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.618+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2790011b00 con 0x7f27981022c0 2024-03-16T10:06:26.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.619+0000 7f2794c00640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 0x7f277c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:26.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.619+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f27900cf2b0 con 0x7f27981022c0 2024-03-16T10:06:26.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.620+0000 7f2797400640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 0x7f277c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:26.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.620+0000 7f2797400640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 0x7f277c0a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f279819b1b0 tx=0x7f278000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:26.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.623+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2790016030 con 0x7f27981022c0 2024-03-16T10:06:26.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3430423677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/775302114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:26 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3379866098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.860+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f279802d1b0 con 0x7f27981022c0 2024-03-16T10:06:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.861+0000 7f2794c00640 1 -- 172.21.15.39:0/2755692131 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f279009c0b0 con 0x7f27981022c0 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3114779354 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3549110681 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1618736720 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/865907722 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4208944392 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3617878948 2024-03-17T09:57:37.234906+0000 2024-03-16T10:06:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:06:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:listed 10 entries 2024-03-16T10:06:26.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3430423677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/775302114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:26 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3379866098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:06:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.863+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 msgr2=0x7f277c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.863+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 0x7f277c0a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f279819b1b0 tx=0x7f278000b390 comp rx=0 tx=0).stop 2024-03-16T10:06:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.863+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 msgr2=0x7f2798199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.863+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f279000eab0 tx=0x7f279000ef70 comp rx=0 tx=0).stop 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 shutdown_connections 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f277c0a2880 0x7f277c0a4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2798107520 0x7f27981a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2798104bf0 0x7f279819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 --2- 172.21.15.39:0/2755692131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27981022c0 0x7f2798199cb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 >> 172.21.15.39:0/2755692131 conn(0x7f27980fbc90 msgr2=0x7f27981018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 shutdown_connections 2024-03-16T10:06:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:26.864+0000 7f279f000640 1 -- 172.21.15.39:0/2755692131 wait complete. 2024-03-16T10:06:26.885 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi039.front.sepia.ceph.com... 2024-03-16T10:06:26.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:26.885 DEBUG:teuthology.orchestra.run.smithi039:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-16T10:06:26.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[75708]: fuse finished with error 0 and tester_r 0 2024-03-16T10:06:26.980 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:06:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:27 smithi194 ceph-mon[36445]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:27 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2755692131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:27 smithi039 ceph-mon[31510]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:27 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2755692131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:27 smithi105 ceph-mon[36559]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:27 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2755692131' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:06:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:29 smithi194 ceph-mon[36445]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:29 smithi039 ceph-mon[31510]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:29 smithi105 ceph-mon[36559]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:31 smithi039 ceph-mon[31510]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:31 smithi105 ceph-mon[36559]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:31 smithi194 ceph-mon[36445]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:06:33.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:06:33.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:06:33.082 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-16T10:06:33.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.386+0000 7efcb8e00640 1 -- 172.21.15.39:0/3914814131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 msgr2=0x7efcb4100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 --2- 172.21.15.39:0/3914814131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb4100c20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c02bc50 comp rx=0 tx=0).stop 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 -- 172.21.15.39:0/3914814131 shutdown_connections 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 --2- 172.21.15.39:0/3914814131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb4101160 0x7efcb41015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 --2- 172.21.15.39:0/3914814131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb4100c20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 --2- 172.21.15.39:0/3914814131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb40ffe60 0x7efcb4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 -- 172.21.15.39:0/3914814131 >> 172.21.15.39:0/3914814131 conn(0x7efcb40fba60 msgr2=0x7efcb40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 -- 172.21.15.39:0/3914814131 shutdown_connections 2024-03-16T10:06:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.387+0000 7efcb8e00640 1 -- 172.21.15.39:0/3914814131 wait complete. 2024-03-16T10:06:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 Processor -- start 2024-03-16T10:06:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 -- start start 2024-03-16T10:06:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb4101160 0x7efcb41a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f4a0 con 0x7efcb41007c0 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f320 con 0x7efcb40ffe60 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.388+0000 7efcb8e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efcb410f620 con 0x7efcb4101160 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33666/0 (socket says 172.21.15.39:33666) 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 -- 172.21.15.39:0/1210071344 learned_addr learned my addr 172.21.15.39:0/1210071344 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb2a00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb4101160 0x7efcb41a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 -- 172.21.15.39:0/1210071344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb4101160 msgr2=0x7efcb41a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb4101160 0x7efcb41a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 -- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 msgr2=0x7efcb419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 -- 172.21.15.39:0/1210071344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb41a16e0 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.389+0000 7efcb3400640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efca400e9f0 tx=0x7efca400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.390+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca400cd80 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.390+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4005500 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.390+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb41a19d0 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.390+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4010690 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.390+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a1ef0 con 0x7efcb40ffe60 2024-03-16T10:06:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.391+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb410ad00 con 0x7efcb40ffe60 2024-03-16T10:06:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.393+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efca40046c0 con 0x7efcb40ffe60 2024-03-16T10:06:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.394+0000 7efcabe00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 0x7efc980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.394+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efca40ceff0 con 0x7efcb40ffe60 2024-03-16T10:06:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.394+0000 7efcb2a00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 0x7efc980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.395+0000 7efcb2a00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 0x7efc980a4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02c260 tx=0x7efc9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.396+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efca409be30 con 0x7efcb40ffe60 2024-03-16T10:06:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:33 smithi039 ceph-mon[31510]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:06:33.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.601+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcb402d380 con 0x7efcb40ffe60 2024-03-16T10:06:33.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.603+0000 7efcabe00640 1 -- 172.21.15.39:0/1210071344 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7efca409b730 con 0x7efcb40ffe60 2024-03-16T10:06:33.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:33.606 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 msgr2=0x7efc980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 0x7efc980a4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02c260 tx=0x7efc9c000bc0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 msgr2=0x7efcb4199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efca400e9f0 tx=0x7efca400edf0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 shutdown_connections 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efc980a2880 0x7efc980a4d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcb4101160 0x7efcb41a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb41007c0 0x7efcb419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 --2- 172.21.15.39:0/1210071344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcb40ffe60 0x7efcb4199ac0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.606+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 >> 172.21.15.39:0/1210071344 conn(0x7efcb40fba60 msgr2=0x7efcb40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:33.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.607+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 shutdown_connections 2024-03-16T10:06:33.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.607+0000 7efcb8e00640 1 -- 172.21.15.39:0/1210071344 wait complete. 2024-03-16T10:06:33.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 -- 172.21.15.39:0/357555905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d0102820 msgr2=0x7f96d0102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 --2- 172.21.15.39:0/357555905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d0102820 0x7f96d0102ca0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009900 tx=0x7f96bc02bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 -- 172.21.15.39:0/357555905 shutdown_connections 2024-03-16T10:06:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 --2- 172.21.15.39:0/357555905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d01031e0 0x7f96d0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 --2- 172.21.15.39:0/357555905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d0102820 0x7f96d0102ca0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 --2- 172.21.15.39:0/357555905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.840+0000 7f96d7000640 1 -- 172.21.15.39:0/357555905 >> 172.21.15.39:0/357555905 conn(0x7f96d00fcbc0 msgr2=0x7f96d00fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.841+0000 7f96d7000640 1 -- 172.21.15.39:0/357555905 shutdown_connections 2024-03-16T10:06:33.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.841+0000 7f96d7000640 1 -- 172.21.15.39:0/357555905 wait complete. 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 Processor -- start 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 -- start start 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 0x7f96d01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96d010d300 con 0x7f96d0102820 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96d010d180 con 0x7f96d01031e0 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f96d010d480 con 0x7f96d0101630 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.842+0000 7f96d6600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 0x7f96d01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49734/0 (socket says 172.21.15.39:49734) 2024-03-16T10:06:33.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d5200640 1 -- 172.21.15.39:0/524433701 learned_addr learned my addr 172.21.15.39:0/524433701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d5c00640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 -- 172.21.15.39:0/524433701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 msgr2=0x7f96d0199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 -- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 msgr2=0x7f96d019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 -- 172.21.15.39:0/524433701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d01a17d0 con 0x7f96d01031e0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d5c00640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d5200640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d6600640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 0x7f96d01a1160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009fd0 tx=0x7f96c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c800ce20 con 0x7f96d01031e0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c8005020 con 0x7f96d01031e0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.843+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d01a1ac0 con 0x7f96d01031e0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.844+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8010720 con 0x7f96d01031e0 2024-03-16T10:06:33.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.844+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01a1fe0 con 0x7f96d01031e0 2024-03-16T10:06:33.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.845+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d01a2270 con 0x7f96d01031e0 2024-03-16T10:06:33.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.846+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f96c8005540 con 0x7f96d01031e0 2024-03-16T10:06:33.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.848+0000 7f96ce000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 0x7f96b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:33.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.848+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f96c80cf360 con 0x7f96d01031e0 2024-03-16T10:06:33.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.848+0000 7f96d5c00640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 0x7f96b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:33.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.849+0000 7f96d5c00640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 0x7f96b80a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f96d0102680 tx=0x7f96c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:33.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:33.852+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f96c809c160 con 0x7f96d01031e0 2024-03-16T10:06:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:33 smithi105 ceph-mon[36559]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:06:33.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:33 smithi194 ceph-mon[36445]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:06:34.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.056+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96d01a2440 con 0x7f96d01031e0 2024-03-16T10:06:34.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.058+0000 7f96ce000640 1 -- 172.21.15.39:0/524433701 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7f96c809ba60 con 0x7f96d01031e0 2024-03-16T10:06:34.060 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:34.060 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 msgr2=0x7f96b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 0x7f96b80a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f96d0102680 tx=0x7f96c4005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 msgr2=0x7f96d01a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 0x7f96d01a1160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009fd0 tx=0x7f96c800ef10 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 shutdown_connections 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f96b80a2880 0x7f96b80a4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96d01031e0 0x7f96d01a1160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d0102820 0x7f96d019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 --2- 172.21.15.39:0/524433701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96d0101630 0x7f96d0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 >> 172.21.15.39:0/524433701 conn(0x7f96d00fcbc0 msgr2=0x7f96d0073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 shutdown_connections 2024-03-16T10:06:34.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.062+0000 7f96d7000640 1 -- 172.21.15.39:0/524433701 wait complete. 2024-03-16T10:06:34.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T10:06:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.314+0000 7fc008400640 1 -- 172.21.15.39:0/386637521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 msgr2=0x7fc000105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.314+0000 7fc008400640 1 --2- 172.21.15.39:0/386637521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc000105070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02bc50 comp rx=0 tx=0).stop 2024-03-16T10:06:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.314+0000 7fc008400640 1 -- 172.21.15.39:0/386637521 shutdown_connections 2024-03-16T10:06:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.314+0000 7fc008400640 1 --2- 172.21.15.39:0/386637521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0001055b0 0x7fc00010be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.315+0000 7fc008400640 1 --2- 172.21.15.39:0/386637521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc000105070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.315+0000 7fc008400640 1 --2- 172.21.15.39:0/386637521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc000103a00 0x7fc000103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.315+0000 7fc008400640 1 -- 172.21.15.39:0/386637521 >> 172.21.15.39:0/386637521 conn(0x7fc0000fef90 msgr2=0x7fc0001013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.315+0000 7fc008400640 1 -- 172.21.15.39:0/386637521 shutdown_connections 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.315+0000 7fc008400640 1 -- 172.21.15.39:0/386637521 wait complete. 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.316+0000 7fc008400640 1 Processor -- start 2024-03-16T10:06:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.316+0000 7fc008400640 1 -- start start 2024-03-16T10:06:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc000103a00 0x7fc000199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0001055b0 0x7fc0001a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010f6b0 con 0x7fc0001055b0 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010f530 con 0x7fc000104bf0 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc008400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010f830 con 0x7fc000103a00 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33708/0 (socket says 172.21.15.39:33708) 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 -- 172.21.15.39:0/3972559819 learned_addr learned my addr 172.21.15.39:0/3972559819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005a00640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc000103a00 0x7fc000199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc006400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0001055b0 0x7fc0001a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 -- 172.21.15.39:0/3972559819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc000103a00 msgr2=0x7fc000199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc000103a00 0x7fc000199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 -- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0001055b0 msgr2=0x7fc0001a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0001055b0 0x7fc0001a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.317+0000 7fc005000640 1 -- 172.21.15.39:0/3972559819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0001a1920 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fc005000640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfec005ab0 tx=0x7fbfec02c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec041070 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfec004450 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec034440 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0001a1bb0 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.318+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001a20b0 con 0x7fc000104bf0 2024-03-16T10:06:34.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.319+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00010acc0 con 0x7fc000104bf0 2024-03-16T10:06:34.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.321+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbfec044070 con 0x7fc000104bf0 2024-03-16T10:06:34.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.322+0000 7fbffe000640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 0x7fbfe80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:34.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.323+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbfec0f09d0 con 0x7fc000104bf0 2024-03-16T10:06:34.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.323+0000 7fc005a00640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 0x7fbfe80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:34.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.323+0000 7fc005a00640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 0x7fbfe80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc00019b1e0 tx=0x7fbff4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:34.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.326+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbfec0bd7d0 con 0x7fc000104bf0 2024-03-16T10:06:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:34.575+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc00002d1b0 con 0x7fc000104bf0 2024-03-16T10:06:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:34 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1210071344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:34.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:34 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/524433701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:34.765 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:34 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1210071344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:34.765 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:34 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/524433701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:34.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:34 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1210071344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:34.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:34 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/524433701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:35.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.488+0000 7fbffe000640 1 -- 172.21.15.39:0/3972559819 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbfec0bd0d0 con 0x7fc000104bf0 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 msgr2=0x7fbfe80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 0x7fbfe80a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc00019b1e0 tx=0x7fbff4005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 msgr2=0x7fc00019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfec005ab0 tx=0x7fbfec02c6e0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 shutdown_connections 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbfe80a2800 0x7fbfe80a4cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0001055b0 0x7fc0001a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc000104bf0 0x7fc00019a250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 --2- 172.21.15.39:0/3972559819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc000103a00 0x7fc000199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 >> 172.21.15.39:0/3972559819 conn(0x7fc0000fef90 msgr2=0x7fc000107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 shutdown_connections 2024-03-16T10:06:35.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.492+0000 7fc008400640 1 -- 172.21.15.39:0/3972559819 wait complete. 2024-03-16T10:06:35.514 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-16T10:06:35.517 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 -- 172.21.15.39:0/374674355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f0100890 msgr2=0x7f21f0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 --2- 172.21.15.39:0/374674355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f0100890 0x7f21f0100cf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009900 tx=0x7f21d802bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 -- 172.21.15.39:0/374674355 shutdown_connections 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 --2- 172.21.15.39:0/374674355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 0x7f21f01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 --2- 172.21.15.39:0/374674355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f0100890 0x7f21f0100cf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 --2- 172.21.15.39:0/374674355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f00fff30 0x7f21f0100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 -- 172.21.15.39:0/374674355 >> 172.21.15.39:0/374674355 conn(0x7f21f00fbb50 msgr2=0x7f21f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:35.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 -- 172.21.15.39:0/374674355 shutdown_connections 2024-03-16T10:06:35.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.739+0000 7f21f5a00640 1 -- 172.21.15.39:0/374674355 wait complete. 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.740+0000 7f21f5a00640 1 Processor -- start 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 -- start start 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f0100890 0x7f21f019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 0x7f21f01a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010f570 con 0x7f21f0101230 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010f3f0 con 0x7f21f00fff30 2024-03-16T10:06:35.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010f6f0 con 0x7f21f0100890 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21efe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 0x7f21f01a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21efe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33730/0 (socket says 172.21.15.39:33730) 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21efe00640 1 -- 172.21.15.39:0/4091218499 learned_addr learned my addr 172.21.15.39:0/4091218499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.741+0000 7f21ef400640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f0100890 0x7f21f019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 -- 172.21.15.39:0/4091218499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f0100890 msgr2=0x7f21f019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f0100890 0x7f21f019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 -- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 msgr2=0x7f21f01a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 0x7f21f01a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 -- 172.21.15.39:0/4091218499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f01a1780 con 0x7f21f00fff30 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21efe00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e9f0 tx=0x7f21e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000cd80 con 0x7f21f00fff30 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e0005500 con 0x7f21f00fff30 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01a1a70 con 0x7f21f00fff30 2024-03-16T10:06:35.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.742+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0010690 con 0x7f21f00fff30 2024-03-16T10:06:35.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.743+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01a1f90 con 0x7f21f00fff30 2024-03-16T10:06:35.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.744+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f010add0 con 0x7f21f00fff30 2024-03-16T10:06:35.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.745+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f21e00046c0 con 0x7f21f00fff30 2024-03-16T10:06:35.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.747+0000 7f21ecc00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 0x7f21d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:35.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.747+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f21e00cf060 con 0x7f21f00fff30 2024-03-16T10:06:35.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.747+0000 7f21ef400640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 0x7f21d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:35.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.748+0000 7f21ef400640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 0x7f21d40a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21d802c240 tx=0x7f21d802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:35.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:35.750+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21e009be60 con 0x7f21f00fff30 2024-03-16T10:06:35.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:35 smithi039 ceph-mon[31510]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:35.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:35 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3972559819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:35.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:35 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:35 smithi105 ceph-mon[36559]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:35 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3972559819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:35.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:35 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:35 smithi194 ceph-mon[36445]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:35 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3972559819' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:35.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:35 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T10:06:36.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.003+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f21f00623e0 con 0x7f21f00fff30 2024-03-16T10:06:36.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.493+0000 7f21ecc00640 1 -- 172.21.15.39:0/4091218499 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f21e009b760 con 0x7f21f00fff30 2024-03-16T10:06:36.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.496+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 msgr2=0x7f21d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:36.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.496+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 0x7f21d40a4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21d802c240 tx=0x7f21d802df90 comp rx=0 tx=0).stop 2024-03-16T10:06:36.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 msgr2=0x7f21f0199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:36.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e9f0 tx=0x7f21e000edf0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 shutdown_connections 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f21d40a2880 0x7f21d40a4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0101230 0x7f21f01a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21f0100890 0x7f21f019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 --2- 172.21.15.39:0/4091218499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21f00fff30 0x7f21f0199b60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 >> 172.21.15.39:0/4091218499 conn(0x7f21f00fbb50 msgr2=0x7f21f00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 shutdown_connections 2024-03-16T10:06:36.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.497+0000 7f21f5a00640 1 -- 172.21.15.39:0/4091218499 wait complete. 2024-03-16T10:06:36.524 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 -- 172.21.15.39:0/2871726323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14941099b0 msgr2=0x7f1494110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 --2- 172.21.15.39:0/2871726323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14941099b0 0x7f1494110240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f148c009be0 tx=0x7f148c02bef0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 -- 172.21.15.39:0/2871726323 shutdown_connections 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 --2- 172.21.15.39:0/2871726323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14941099b0 0x7f1494110240 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 --2- 172.21.15.39:0/2871726323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 0x7f1494109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 --2- 172.21.15.39:0/2871726323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1494107e00 0x7f1494108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 -- 172.21.15.39:0/2871726323 >> 172.21.15.39:0/2871726323 conn(0x7f1494076da0 msgr2=0x7f14940791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 -- 172.21.15.39:0/2871726323 shutdown_connections 2024-03-16T10:06:36.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.717+0000 7f149c600640 1 -- 172.21.15.39:0/2871726323 wait complete. 2024-03-16T10:06:36.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.718+0000 7f149c600640 1 Processor -- start 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.718+0000 7f149c600640 1 -- start start 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 0x7f149419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14941099b0 0x7f14941a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1494113a80 con 0x7f14941099b0 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1494113900 con 0x7f1494107e00 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149c600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1494113c00 con 0x7f1494108ff0 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f149a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14941099b0 0x7f14941a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33742/0 (socket says 172.21.15.39:33742) 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 -- 172.21.15.39:0/2965340734 learned_addr learned my addr 172.21.15.39:0/2965340734 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499200640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 0x7f149419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:36.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 -- 172.21.15.39:0/2965340734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 msgr2=0x7f149419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 0x7f149419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 -- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14941099b0 msgr2=0x7f14941a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14941099b0 0x7f14941a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.719+0000 7f1499c00640 1 -- 172.21.15.39:0/2965340734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14941a5cb0 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f1499c00640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f148800e9f0 tx=0x7f148800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800ccc0 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14880035f0 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800ce60 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14941a5fa0 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.720+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14941a63b0 con 0x7f1494107e00 2024-03-16T10:06:36.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.721+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149410f120 con 0x7f1494107e00 2024-03-16T10:06:36.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.723+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f14880037a0 con 0x7f1494107e00 2024-03-16T10:06:36.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.724+0000 7f1492000640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 0x7f147c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:36.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.724+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f14880ceee0 con 0x7f1494107e00 2024-03-16T10:06:36.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.724+0000 7f1499200640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 0x7f147c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:36.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.725+0000 7f1499200640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 0x7f147c0a4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f149419f5a0 tx=0x7f1484005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:36.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.733+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f148809bce0 con 0x7f1494107e00 2024-03-16T10:06:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:36 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T10:06:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:36 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:36 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4091218499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:36 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:36 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T10:06:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:36 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:36 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4091218499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:36 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:36 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T10:06:36.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:36 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:36.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:36 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4091218499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:36 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-16T10:06:36.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:36.978+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f1494062480 con 0x7f1494107e00 2024-03-16T10:06:37.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.519+0000 7f1492000640 1 -- 172.21.15.39:0/2965340734 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f148809b5e0 con 0x7f1494107e00 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 msgr2=0x7f147c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 0x7f147c0a4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f149419f5a0 tx=0x7f1484005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 msgr2=0x7f149419e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f148800e9f0 tx=0x7f148800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 shutdown_connections 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f147c0a2880 0x7f147c0a4d40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14941099b0 0x7f14941a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1494108ff0 0x7f149419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 --2- 172.21.15.39:0/2965340734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1494107e00 0x7f149419e090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 >> 172.21.15.39:0/2965340734 conn(0x7f1494076da0 msgr2=0x7f149410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 shutdown_connections 2024-03-16T10:06:37.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.523+0000 7f149c600640 1 -- 172.21.15.39:0/2965340734 wait complete. 2024-03-16T10:06:37.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-16T10:06:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.747+0000 7f7333200640 1 -- 172.21.15.39:0/226440728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 msgr2=0x7f7334109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 --2- 172.21.15.39:0/226440728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334109b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f732c009ca0 tx=0x7f732c02c0f0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 -- 172.21.15.39:0/226440728 shutdown_connections 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 --2- 172.21.15.39:0/226440728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334109b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 --2- 172.21.15.39:0/226440728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341028d0 0x7f7334102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 --2- 172.21.15.39:0/226440728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341016e0 0x7f7334101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 -- 172.21.15.39:0/226440728 >> 172.21.15.39:0/226440728 conn(0x7f73340fcc50 msgr2=0x7f73340ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 -- 172.21.15.39:0/226440728 shutdown_connections 2024-03-16T10:06:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.748+0000 7f7333200640 1 -- 172.21.15.39:0/226440728 wait complete. 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 Processor -- start 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 -- start start 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341016e0 0x7f7334075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 0x7f7334073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f733410d360 con 0x7f73341028d0 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f733410d1e0 con 0x7f7334103290 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7333200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f733410d4e0 con 0x7f73341016e0 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7331400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 0x7f7334073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7332800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7332800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33764/0 (socket says 172.21.15.39:33764) 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7331400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 0x7f7334073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49784/0 (socket says 172.21.15.39:49784) 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7332800640 1 -- 172.21.15.39:0/3088231778 learned_addr learned my addr 172.21.15.39:0/3088231778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.749+0000 7f7331e00640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341016e0 0x7f7334075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 -- 172.21.15.39:0/3088231778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341016e0 msgr2=0x7f7334075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341016e0 0x7f7334075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 -- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 msgr2=0x7f7334073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 0x7f7334073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 -- 172.21.15.39:0/3088231778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73340749a0 con 0x7f7334103290 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7332800640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f732c037040 tx=0x7f732c032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c041020 con 0x7f7334103290 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c034cd0 con 0x7f7334103290 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a1a00 con 0x7f7334103290 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.750+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c039040 con 0x7f7334103290 2024-03-16T10:06:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.751+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341a1f50 con 0x7f7334103290 2024-03-16T10:06:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.752+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334108a00 con 0x7f7334103290 2024-03-16T10:06:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.753+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f732c044020 con 0x7f7334103290 2024-03-16T10:06:37.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.755+0000 7f732aa00640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 0x7f73180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:37.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.755+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f732c0f0b90 con 0x7f7334103290 2024-03-16T10:06:37.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.755+0000 7f7331e00640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 0x7f73180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:37.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.756+0000 7f7331e00640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 0x7f73180a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7334102730 tx=0x7f7324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:37.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:37.758+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f732c04b020 con 0x7f7334103290 2024-03-16T10:06:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2965340734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:37.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:37 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2965340734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:37.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:37 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-03-16T10:06:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T10:06:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-16T10:06:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T10:06:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T10:06:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-16T10:06:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-16T10:06:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2965340734' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:37 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T10:06:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.010+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f73340623e0 con 0x7f7334103290 2024-03-16T10:06:38.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.534+0000 7f732aa00640 1 -- 172.21.15.39:0/3088231778 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f732c0bda80 con 0x7f7334103290 2024-03-16T10:06:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 msgr2=0x7f73180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 0x7f73180a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7334102730 tx=0x7f7324005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 msgr2=0x7f7334074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f732c037040 tx=0x7f732c032f40 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 shutdown_connections 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f73180a2800 0x7f73180a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7334103290 0x7f7334074110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73341028d0 0x7f7334073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 --2- 172.21.15.39:0/3088231778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73341016e0 0x7f7334075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.539+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 >> 172.21.15.39:0/3088231778 conn(0x7f73340fcc50 msgr2=0x7f7334105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:38.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.540+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 shutdown_connections 2024-03-16T10:06:38.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.540+0000 7f7333200640 1 -- 172.21.15.39:0/3088231778 wait complete. 2024-03-16T10:06:38.571 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-16T10:06:38.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 -- 172.21.15.39:0/124381070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280105490 msgr2=0x7f528010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 --2- 172.21.15.39:0/124381070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280105490 0x7f528010bd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5278009860 tx=0x7f527802bb70 comp rx=0 tx=0).stop 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 -- 172.21.15.39:0/124381070 shutdown_connections 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 --2- 172.21.15.39:0/124381070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280105490 0x7f528010bd20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 --2- 172.21.15.39:0/124381070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5280104ad0 0x7f5280104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 --2- 172.21.15.39:0/124381070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52801038e0 0x7f5280103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 -- 172.21.15.39:0/124381070 >> 172.21.15.39:0/124381070 conn(0x7f52800fee50 msgr2=0x7f5280101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 -- 172.21.15.39:0/124381070 shutdown_connections 2024-03-16T10:06:38.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.767+0000 7f5286000640 1 -- 172.21.15.39:0/124381070 wait complete. 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.768+0000 7f5286000640 1 Processor -- start 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 -- start start 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 0x7f5280199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 0x7f52801a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f528010f540 con 0x7f52801038e0 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f528010f3c0 con 0x7f5280105490 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5286000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f528010f6c0 con 0x7f5280104ad0 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5284c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 0x7f5280199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:38.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f527fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5284c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 0x7f5280199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49810/0 (socket says 172.21.15.39:49810) 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f527fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:44726/0 (socket says 172.21.15.39:44726) 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5284c00640 1 -- 172.21.15.39:0/1919503513 learned_addr learned my addr 172.21.15.39:0/1919503513 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.769+0000 7f5285600640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 0x7f52801a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 -- 172.21.15.39:0/1919503513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 msgr2=0x7f52801a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 0x7f52801a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 -- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 msgr2=0x7f5280199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 0x7f5280199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 -- 172.21.15.39:0/1919503513 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52801a1780 con 0x7f5280104ad0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f5285600640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 0x7f52801a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527fe00640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e9f0 tx=0x7f526c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c00cd80 con 0x7f5280104ad0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526c0035f0 con 0x7f5280104ad0 2024-03-16T10:06:38.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801a1a70 con 0x7f5280104ad0 2024-03-16T10:06:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c010690 con 0x7f5280104ad0 2024-03-16T10:06:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.770+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a2010 con 0x7f5280104ad0 2024-03-16T10:06:38.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.771+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528010ac20 con 0x7f5280104ad0 2024-03-16T10:06:38.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.774+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f526c003b30 con 0x7f5280104ad0 2024-03-16T10:06:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.775+0000 7f527d600640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 0x7f52680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.776+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f526c0cef30 con 0x7f5280104ad0 2024-03-16T10:06:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.776+0000 7f5284c00640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 0x7f52680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:38.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.776+0000 7f5284c00640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 0x7f52680a4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5274009a60 tx=0x7f5274010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:38.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:38.784+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f526c09bd30 con 0x7f5280104ad0 2024-03-16T10:06:38.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:active 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3088231778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:38.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:38 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:active 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3088231778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:38.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:38 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:39.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T10:06:39.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] up:active 2024-03-16T10:06:39.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T10:06:39.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-03-16T10:06:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-03-16T10:06:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-16T10:06:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3088231778' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:39.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:38 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T10:06:39.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.049+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5280061c20 con 0x7f52680a2880 2024-03-16T10:06:39.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:06:39.058+0000 7ff91c400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:06:39.506 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:06:39.058+0000 7ff91c400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:06:39.506 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:06:39.058+0000 7ff91c400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:06:39.506 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:06:39.058+0000 7ff91c400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:06:39.506 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:06:39.058+0000 7ff91c400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:06:39.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.582+0000 7f527d600640 1 -- 172.21.15.39:0/1919503513 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5280061c20 con 0x7f52680a2880 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.585+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 msgr2=0x7f52680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.585+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 0x7f52680a4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5274009a60 tx=0x7f5274010040 comp rx=0 tx=0).stop 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 msgr2=0x7f528019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f526c00e9f0 tx=0x7f526c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 shutdown_connections 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52680a2880 0x7f52680a4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5280105490 0x7f52801a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280104ad0 0x7f528019a0a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 --2- 172.21.15.39:0/1919503513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52801038e0 0x7f5280199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 >> 172.21.15.39:0/1919503513 conn(0x7f52800fee50 msgr2=0x7f5280107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 shutdown_connections 2024-03-16T10:06:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.586+0000 7f5286000640 1 -- 172.21.15.39:0/1919503513 wait complete. 2024-03-16T10:06:39.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:06:39.289+0000 7f8ac9e70700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:06:39.289+0000 7f8ac9e70700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:active 2024-03-16T10:06:39.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-16T10:06:39.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-16T10:06:39.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-16T10:06:39.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-16T10:06:39.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:39 smithi105 ceph-mon[36559]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 -- 172.21.15.39:0/97346950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 msgr2=0x7fba4c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 --2- 172.21.15.39:0/97346950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c1003b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba40009900 tx=0x7fba4002bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 -- 172.21.15.39:0/97346950 shutdown_connections 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 --2- 172.21.15.39:0/97346950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 --2- 172.21.15.39:0/97346950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 --2- 172.21.15.39:0/97346950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c1003b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 -- 172.21.15.39:0/97346950 >> 172.21.15.39:0/97346950 conn(0x7fba4c0fbb50 msgr2=0x7fba4c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 -- 172.21.15.39:0/97346950 shutdown_connections 2024-03-16T10:06:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.821+0000 7fba52800640 1 -- 172.21.15.39:0/97346950 wait complete. 2024-03-16T10:06:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.822+0000 7fba52800640 1 Processor -- start 2024-03-16T10:06:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 -- start start 2024-03-16T10:06:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f550 con 0x7fba4c1008f0 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f3d0 con 0x7fba4c0fff90 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba52800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c10f6d0 con 0x7fba4c101290 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba51400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba50a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba51e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.823+0000 7fba51400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:33816/0 (socket says 172.21.15.39:33816) 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 -- 172.21.15.39:0/3643387689 learned_addr learned my addr 172.21.15.39:0/3643387689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 -- 172.21.15.39:0/3643387689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 msgr2=0x7fba4c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 -- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 msgr2=0x7fba4c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 -- 172.21.15.39:0/3643387689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c1a17e0 con 0x7fba4c0fff90 2024-03-16T10:06:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba50a00640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51e00640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.824+0000 7fba51400640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba40000c00 tx=0x7fba4002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.825+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40043020 con 0x7fba4c0fff90 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.825+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba400054e0 con 0x7fba4c0fff90 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.825+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40003cd0 con 0x7fba4c0fff90 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.825+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c1a1a70 con 0x7fba4c0fff90 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.825+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c1a1f70 con 0x7fba4c0fff90 2024-03-16T10:06:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.826+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c10ae20 con 0x7fba4c0fff90 2024-03-16T10:06:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.828+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fba40046020 con 0x7fba4c0fff90 2024-03-16T10:06:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.830+0000 7fba4a000640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 0x7fba340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.830+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fba400f0cc0 con 0x7fba4c0fff90 2024-03-16T10:06:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.830+0000 7fba50a00640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 0x7fba340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.831+0000 7fba50a00640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 0x7fba340a4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba4c19b0d0 tx=0x7fba38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:39.833+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba400f7080 con 0x7fba4c0fff90 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:active 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-16T10:06:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:39 smithi194 ceph-mon[36445]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: pgmap v333: 129 pgs: 129 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-03-16T10:06:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] up:active 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-16T10:06:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:39 smithi039 ceph-mon[31510]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.066+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fba4c061cb0 con 0x7fba340a2880 2024-03-16T10:06:40.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.098+0000 7fba4a000640 1 -- 172.21.15.39:0/3643387689 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba4c061cb0 con 0x7fba340a2880 2024-03-16T10:06:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 msgr2=0x7fba340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 0x7fba340a4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba4c19b0d0 tx=0x7fba38005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 msgr2=0x7fba4c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba40000c00 tx=0x7fba4002c650 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 shutdown_connections 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fba340a2880 0x7fba340a4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba4c101290 0x7fba4c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba4c1008f0 0x7fba4c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 --2- 172.21.15.39:0/3643387689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba4c0fff90 0x7fba4c199bc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 >> 172.21.15.39:0/3643387689 conn(0x7fba4c0fbb50 msgr2=0x7fba4c0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 shutdown_connections 2024-03-16T10:06:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.101+0000 7fba52800640 1 -- 172.21.15.39:0/3643387689 wait complete. 2024-03-16T10:06:40.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-16T10:06:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.354+0000 7f1649800640 1 -- 172.21.15.39:0/2486995488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16440757e0 msgr2=0x7f1644073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.354+0000 7f1649800640 1 --2- 172.21.15.39:0/2486995488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16440757e0 0x7f1644073c30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1634009ca0 tx=0x7f163402c100 comp rx=0 tx=0).stop 2024-03-16T10:06:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 -- 172.21.15.39:0/2486995488 shutdown_connections 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 --2- 172.21.15.39:0/2486995488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f164410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 --2- 172.21.15.39:0/2486995488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1644074200 0x7f1644074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 --2- 172.21.15.39:0/2486995488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16440757e0 0x7f1644073c30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 -- 172.21.15.39:0/2486995488 >> 172.21.15.39:0/2486995488 conn(0x7f16440fcc50 msgr2=0x7f16440ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 -- 172.21.15.39:0/2486995488 shutdown_connections 2024-03-16T10:06:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.355+0000 7f1649800640 1 -- 172.21.15.39:0/2486995488 wait complete. 2024-03-16T10:06:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 Processor -- start 2024-03-16T10:06:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 -- start start 2024-03-16T10:06:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440757e0 0x7f164419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f164410f520 con 0x7f16440757e0 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f164410f3a0 con 0x7f1644109590 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1649800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f164410f6a0 con 0x7f1644074200 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1643400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440757e0 0x7f164419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1648e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1643e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1648e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40742/0 (socket says 172.21.15.39:40742) 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1643e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47652/0 (socket says 172.21.15.39:47652) 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.356+0000 7f1648e00640 1 -- 172.21.15.39:0/1971709408 learned_addr learned my addr 172.21.15.39:0/1971709408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 -- 172.21.15.39:0/1971709408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 msgr2=0x7f16441a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 -- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440757e0 msgr2=0x7f164419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440757e0 0x7f164419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 -- 172.21.15.39:0/1971709408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441a17e0 con 0x7f1644074200 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1648e00640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.357+0000 7f1643e00640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1634009bf0 tx=0x7f1634035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.358+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634039040 con 0x7f1644074200 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.358+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1634003970 con 0x7f1644074200 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.358+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16441a1a70 con 0x7f1644074200 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.358+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634004360 con 0x7f1644074200 2024-03-16T10:06:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.358+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441a1f30 con 0x7f1644074200 2024-03-16T10:06:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.359+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1644108310 con 0x7f1644074200 2024-03-16T10:06:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.363+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1634046030 con 0x7f1644074200 2024-03-16T10:06:40.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.364+0000 7f1640c00640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 0x7f16280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.364+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f16340f0880 con 0x7f1644074200 2024-03-16T10:06:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.364+0000 7f1643400640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 0x7f16280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.365+0000 7f1643400640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 0x7f16280a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f164419b0d0 tx=0x7f162c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.368+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f163404d060 con 0x7f1644074200 2024-03-16T10:06:40.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.585+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f16340b9680 con 0x7f1644074200 2024-03-16T10:06:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.640+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1644061c20 con 0x7f16280a2800 2024-03-16T10:06:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.644+0000 7f1640c00640 1 -- 172.21.15.39:0/1971709408 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1644061c20 con 0x7f16280a2800 2024-03-16T10:06:40.646 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_0/377a377e-1b7a-4346-9357-e88736a18e57 2024-03-16T10:06:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 msgr2=0x7f16280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 0x7f16280a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f164419b0d0 tx=0x7f162c005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 msgr2=0x7f1644199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1634009bf0 tx=0x7f1634035d10 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 shutdown_connections 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f16280a2800 0x7f16280a4cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1644109590 0x7f16441a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440757e0 0x7f164419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 --2- 172.21.15.39:0/1971709408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1644074200 0x7f1644199bc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 >> 172.21.15.39:0/1971709408 conn(0x7f16440fcc50 msgr2=0x7f16441054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 shutdown_connections 2024-03-16T10:06:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.647+0000 7f1649800640 1 -- 172.21.15.39:0/1971709408 wait complete. 2024-03-16T10:06:40.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-16T10:06:40.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:active 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-16T10:06:40.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:40 smithi105 ceph-mon[36559]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.873+0000 7ff4ba600640 1 -- 172.21.15.39:0/2273597294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4104a70 msgr2=0x7ff4b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.873+0000 7ff4ba600640 1 --2- 172.21.15.39:0/2273597294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4104a70 0x7ff4b4106e60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009900 tx=0x7ff4a002bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 -- 172.21.15.39:0/2273597294 shutdown_connections 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 --2- 172.21.15.39:0/2273597294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 --2- 172.21.15.39:0/2273597294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4104a70 0x7ff4b4106e60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 --2- 172.21.15.39:0/2273597294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4102140 0x7ff4b4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 -- 172.21.15.39:0/2273597294 >> 172.21.15.39:0/2273597294 conn(0x7ff4b40fbb50 msgr2=0x7ff4b40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 -- 172.21.15.39:0/2273597294 shutdown_connections 2024-03-16T10:06:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.874+0000 7ff4ba600640 1 -- 172.21.15.39:0/2273597294 wait complete. 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 Processor -- start 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 -- start start 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4104a70 0x7ff4b419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d380 con 0x7ff4b4104a70 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d200 con 0x7ff4b4102140 2024-03-16T10:06:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.875+0000 7ff4ba600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b410d500 con 0x7ff4b41073a0 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40764/0 (socket says 172.21.15.39:40764) 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 -- 172.21.15.39:0/3356446384 learned_addr learned my addr 172.21.15.39:0/3356446384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9c00640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b3e00640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4104a70 0x7ff4b419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 -- 172.21.15.39:0/3356446384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 msgr2=0x7ff4b41a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 -- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4104a70 msgr2=0x7ff4b419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4104a70 0x7ff4b419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 -- 172.21.15.39:0/3356446384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b41a17f0 con 0x7ff4b4102140 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9c00640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.876+0000 7ff4b9200640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800e9f0 tx=0x7ff4a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.877+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800cd80 con 0x7ff4b4102140 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.877+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8010910 con 0x7ff4b4102140 2024-03-16T10:06:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.877+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8018990 con 0x7ff4b4102140 2024-03-16T10:06:40.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.877+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b41a1ae0 con 0x7ff4b4102140 2024-03-16T10:06:40.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.877+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b41a1fb0 con 0x7ff4b4102140 2024-03-16T10:06:40.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.878+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b4108ef0 con 0x7ff4b4102140 2024-03-16T10:06:40.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.880+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff4a8010ac0 con 0x7ff4b4102140 2024-03-16T10:06:40.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.881+0000 7ff4b1600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 0x7ff49c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:40.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.881+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff4a80cf140 con 0x7ff4b4102140 2024-03-16T10:06:40.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.881+0000 7ff4b3e00640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 0x7ff49c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:40.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.882+0000 7ff4b3e00640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 0x7ff49c0a4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002c170 tx=0x7ff4a002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:40.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:40.885+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff4a809b790 con 0x7ff4b4102140 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:active 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-16T10:06:40.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:40 smithi194 ceph-mon[36445]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] up:active 2024-03-16T10:06:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-16T10:06:41.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T10:06:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:06:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-03-16T10:06:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-16T10:06:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-16T10:06:41.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:40 smithi039 ceph-mon[31510]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:41.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.182+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff4b4061c20 con 0x7ff49c0a2910 2024-03-16T10:06:41.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.212+0000 7ff4b1600640 1 -- 172.21.15.39:0/3356446384 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4b4061c20 con 0x7ff49c0a2910 2024-03-16T10:06:41.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.214+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 msgr2=0x7ff49c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.214+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 0x7ff49c0a4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002c170 tx=0x7ff4a002df90 comp rx=0 tx=0).stop 2024-03-16T10:06:41.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 msgr2=0x7ff4b4199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800e9f0 tx=0x7ff4a800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 shutdown_connections 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff49c0a2910 0x7ff49c0a4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4b41073a0 0x7ff4b41a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4b4104a70 0x7ff4b419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 --2- 172.21.15.39:0/3356446384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4b4102140 0x7ff4b4199bd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 >> 172.21.15.39:0/3356446384 conn(0x7ff4b40fbb50 msgr2=0x7ff4b41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 shutdown_connections 2024-03-16T10:06:41.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.215+0000 7ff4ba600640 1 -- 172.21.15.39:0/3356446384 wait complete. 2024-03-16T10:06:41.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.443+0000 7f11d7e00640 1 -- 172.21.15.39:0/1065075229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 msgr2=0x7f11d8101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.443+0000 7f11d7e00640 1 --2- 172.21.15.39:0/1065075229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8101b00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009900 tx=0x7f11c802bc30 comp rx=0 tx=0).stop 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 -- 172.21.15.39:0/1065075229 shutdown_connections 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 --2- 172.21.15.39:0/1065075229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 0x7f11d8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 --2- 172.21.15.39:0/1065075229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 --2- 172.21.15.39:0/1065075229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8101b00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 -- 172.21.15.39:0/1065075229 >> 172.21.15.39:0/1065075229 conn(0x7f11d80fcc50 msgr2=0x7f11d80ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 -- 172.21.15.39:0/1065075229 shutdown_connections 2024-03-16T10:06:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.444+0000 7f11d7e00640 1 -- 172.21.15.39:0/1065075229 wait complete. 2024-03-16T10:06:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 Processor -- start 2024-03-16T10:06:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 -- start start 2024-03-16T10:06:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 0x7f11d8074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810d3a0 con 0x7f11d8103290 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810d220 con 0x7f11d81016e0 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.445+0000 7f11d7e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810d520 con 0x7f11d81028d0 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11cfe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47698/0 (socket says 172.21.15.39:47698) 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11cfe00640 1 -- 172.21.15.39:0/127309603 learned_addr learned my addr 172.21.15.39:0/127309603 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40772/0 (socket says 172.21.15.39:40772) 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d7400640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 0x7f11d8074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 -- 172.21.15.39:0/127309603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 msgr2=0x7f11d8073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 -- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 msgr2=0x7f11d8074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 0x7f11d8074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 -- 172.21.15.39:0/127309603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d80749a0 con 0x7f11d81016e0 2024-03-16T10:06:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11d6a00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009230 tx=0x7f11c802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.446+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8043020 con 0x7f11d81016e0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.447+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d81a1b60 con 0x7f11d81016e0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.447+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c80054e0 con 0x7f11d81016e0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.447+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8003b60 con 0x7f11d81016e0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.447+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81a20d0 con 0x7f11d81016e0 2024-03-16T10:06:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.448+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d8108a50 con 0x7f11d81016e0 2024-03-16T10:06:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.450+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f11c8046020 con 0x7f11d81016e0 2024-03-16T10:06:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.452+0000 7f11cf400640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 0x7f11bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.452+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f11c80f0df0 con 0x7f11d81016e0 2024-03-16T10:06:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.452+0000 7f11cfe00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 0x7f11bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.453+0000 7f11cfe00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 0x7f11bc0a4d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f11d8075300 tx=0x7f11c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:41.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.455+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f11c804d020 con 0x7f11d81016e0 2024-03-16T10:06:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.693+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f11d8061c20 con 0x7f11bc0a2890 2024-03-16T10:06:41.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.699+0000 7f11cf400640 1 -- 172.21.15.39:0/127309603 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f11d8061c20 con 0x7f11bc0a2890 2024-03-16T10:06:41.700 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_1/700e2c66-fa54-4588-bbc8-89eda93b2cfd 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.701+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 msgr2=0x7f11bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.701+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 0x7f11bc0a4d50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f11d8075300 tx=0x7f11c0005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.701+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 msgr2=0x7f11d8075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.701+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009230 tx=0x7f11c802c5d0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 shutdown_connections 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f11bc0a2890 0x7f11bc0a4d50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11d8103290 0x7f11d8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11d81028d0 0x7f11d8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 --2- 172.21.15.39:0/127309603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d81016e0 0x7f11d8075590 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 >> 172.21.15.39:0/127309603 conn(0x7f11d80fcc50 msgr2=0x7f11d8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 shutdown_connections 2024-03-16T10:06:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.702+0000 7f11d7e00640 1 -- 172.21.15.39:0/127309603 wait complete. 2024-03-16T10:06:41.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:06:41.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-03-16T10:06:41.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:active 2024-03-16T10:06:41.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-16T10:06:41.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-16T10:06:41.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:06:41.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:41 smithi105 ceph-mon[36559]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:active 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:06:41.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:41 smithi194 ceph-mon[36445]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.971+0000 7faebee00640 1 -- 172.21.15.39:0/327922157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8107390 msgr2=0x7faeb8109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.971+0000 7faebee00640 1 --2- 172.21.15.39:0/327922157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8107390 0x7faeb8109780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009ca0 tx=0x7faeb002c100 comp rx=0 tx=0).stop 2024-03-16T10:06:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 -- 172.21.15.39:0/327922157 shutdown_connections 2024-03-16T10:06:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 --2- 172.21.15.39:0/327922157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8107390 0x7faeb8109780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 --2- 172.21.15.39:0/327922157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8104a60 0x7faeb8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 --2- 172.21.15.39:0/327922157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8102130 0x7faeb8104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 -- 172.21.15.39:0/327922157 >> 172.21.15.39:0/327922157 conn(0x7faeb80fba60 msgr2=0x7faeb80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 -- 172.21.15.39:0/327922157 shutdown_connections 2024-03-16T10:06:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.972+0000 7faebee00640 1 -- 172.21.15.39:0/327922157 wait complete. 2024-03-16T10:06:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 Processor -- start 2024-03-16T10:06:41.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 -- start start 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8102130 0x7faeb8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8107390 0x7faeb81a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.973+0000 7faebee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faeb810d280 con 0x7faeb8102130 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebee00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faeb810d100 con 0x7faeb8107390 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebee00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faeb810d400 con 0x7faeb8104a60 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47710/0 (socket says 172.21.15.39:47710) 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 -- 172.21.15.39:0/3256445842 learned_addr learned my addr 172.21.15.39:0/3256445842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebda00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8102130 0x7faeb8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebe400640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8107390 0x7faeb81a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 -- 172.21.15.39:0/3256445842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8107390 msgr2=0x7faeb81a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8107390 0x7faeb81a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 -- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8102130 msgr2=0x7faeb8199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8102130 0x7faeb8199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 -- 172.21.15.39:0/3256445842 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb81a1670 con 0x7faeb8104a60 2024-03-16T10:06:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.974+0000 7faebd000640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9c0 tx=0x7faea400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.975+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea400cd80 con 0x7faeb8104a60 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.975+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea4005500 con 0x7faeb8104a60 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.975+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb81a1960 con 0x7faeb8104a60 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.975+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4010690 con 0x7faeb8104a60 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.975+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb81a1f00 con 0x7faeb8104a60 2024-03-16T10:06:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.976+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb8108e80 con 0x7faeb8104a60 2024-03-16T10:06:41.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.978+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7faea40046c0 con 0x7faeb8104a60 2024-03-16T10:06:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.980+0000 7faeb6000640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 0x7faea00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.980+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faea40cf110 con 0x7faeb8104a60 2024-03-16T10:06:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.980+0000 7faebda00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 0x7faea00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:41.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.981+0000 7faebda00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 0x7faea00a4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faeac0095f0 tx=0x7faeac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:41.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:41.984+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faea409be10 con 0x7faeb8104a60 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: pgmap v334: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] up:active 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:06:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:41 smithi039 ceph-mon[31510]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.243+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeb802d1b0 con 0x7faeb8104a60 2024-03-16T10:06:42.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.244+0000 7faeb6000640 1 -- 172.21.15.39:0/3256445842 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7faea409b710 con 0x7faeb8104a60 2024-03-16T10:06:42.246 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:42.246 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:06:42.247 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:06:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.247+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 msgr2=0x7faea00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.247+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 0x7faea00a4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faeac0095f0 tx=0x7faeac005f90 comp rx=0 tx=0).stop 2024-03-16T10:06:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.247+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 msgr2=0x7faeb8199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9c0 tx=0x7faea400ee80 comp rx=0 tx=0).stop 2024-03-16T10:06:42.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 shutdown_connections 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7faea00a2910 0x7faea00a4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faeb8107390 0x7faeb81a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faeb8104a60 0x7faeb8199f90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 --2- 172.21.15.39:0/3256445842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb8102130 0x7faeb8199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 >> 172.21.15.39:0/3256445842 conn(0x7faeb80fba60 msgr2=0x7faeb8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 shutdown_connections 2024-03-16T10:06:42.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.248+0000 7faebee00640 1 -- 172.21.15.39:0/3256445842 wait complete. 2024-03-16T10:06:42.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:06:42.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 -- 172.21.15.39:0/2022142252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 msgr2=0x7fd284100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 --2- 172.21.15.39:0/2022142252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284100910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd278009980 tx=0x7fd27802de70 comp rx=0 tx=0).stop 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 -- 172.21.15.39:0/2022142252 shutdown_connections 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 --2- 172.21.15.39:0/2022142252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841020a0 0x7fd284108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 --2- 172.21.15.39:0/2022142252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841016e0 0x7fd284101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 --2- 172.21.15.39:0/2022142252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284100910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 -- 172.21.15.39:0/2022142252 >> 172.21.15.39:0/2022142252 conn(0x7fd2840fba60 msgr2=0x7fd2840fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 -- 172.21.15.39:0/2022142252 shutdown_connections 2024-03-16T10:06:42.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.504+0000 7fd28a600640 1 -- 172.21.15.39:0/2022142252 wait complete. 2024-03-16T10:06:42.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.505+0000 7fd28a600640 1 Processor -- start 2024-03-16T10:06:42.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.505+0000 7fd28a600640 1 -- start start 2024-03-16T10:06:42.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841020a0 0x7fd2841a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410b0c0 con 0x7fd2841020a0 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410af40 con 0x7fd2841004f0 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd28a600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd28410b240 con 0x7fd2841016e0 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd289200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd283e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd283e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47722/0 (socket says 172.21.15.39:47722) 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd283e00640 1 -- 172.21.15.39:0/1148885575 learned_addr learned my addr 172.21.15.39:0/1148885575 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd289200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40816/0 (socket says 172.21.15.39:40816) 2024-03-16T10:06:42.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.506+0000 7fd289c00640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841020a0 0x7fd2841a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 -- 172.21.15.39:0/1148885575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 msgr2=0x7fd284199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 -- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841020a0 msgr2=0x7fd2841a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841020a0 0x7fd2841a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 -- 172.21.15.39:0/1148885575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2841a16c0 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd283e00640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d950 tx=0x7fd27000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270017070 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270009c30 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27000f6b0 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.507+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841a19b0 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.508+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841a1f00 con 0x7fd2841016e0 2024-03-16T10:06:42.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.509+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2841077b0 con 0x7fd2841016e0 2024-03-16T10:06:42.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.510+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd2700109d0 con 0x7fd2841016e0 2024-03-16T10:06:42.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.512+0000 7fd281600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 0x7fd26c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.512+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd2700cea00 con 0x7fd2841016e0 2024-03-16T10:06:42.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.512+0000 7fd289200640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 0x7fd26c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.513+0000 7fd289200640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 0x7fd26c0a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd27802e3b0 tx=0x7fd27802ff90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:42.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.516+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd270009de0 con 0x7fd2841016e0 2024-03-16T10:06:42.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.730+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28402d380 con 0x7fd2841016e0 2024-03-16T10:06:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.732+0000 7fd281600640 1 -- 172.21.15.39:0/1148885575 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7fd27009b050 con 0x7fd2841016e0 2024-03-16T10:06:42.734 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:42.734 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:42.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 msgr2=0x7fd26c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 0x7fd26c0a4d50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd27802e3b0 tx=0x7fd27802ff90 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 msgr2=0x7fd284199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d950 tx=0x7fd27000de10 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 shutdown_connections 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd26c0a2890 0x7fd26c0a4d50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2841020a0 0x7fd2841a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2841016e0 0x7fd284199fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 --2- 172.21.15.39:0/1148885575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2841004f0 0x7fd284199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 >> 172.21.15.39:0/1148885575 conn(0x7fd2840fba60 msgr2=0x7fd284104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 shutdown_connections 2024-03-16T10:06:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.735+0000 7fd28a600640 1 -- 172.21.15.39:0/1148885575 wait complete. 2024-03-16T10:06:42.756 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T10:06:42.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:06:42.840 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:42 smithi039 ceph-mon[31510]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.840 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:42 smithi039 ceph-mon[31510]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.840 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:42 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3256445842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:42.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:42 smithi105 ceph-mon[36559]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:42 smithi105 ceph-mon[36559]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:42 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3256445842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:42.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:42 smithi194 ceph-mon[36445]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:42 smithi194 ceph-mon[36445]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:06:42.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:42 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3256445842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:06:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 -- 172.21.15.39:0/2394367151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 msgr2=0x7f0ce4109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/2394367151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce4109930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009900 tx=0x7f0cdc02bc40 comp rx=0 tx=0).stop 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 -- 172.21.15.39:0/2394367151 shutdown_connections 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/2394367151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce4109930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/2394367151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce4104c10 0x7f0ce4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/2394367151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce41022e0 0x7f0ce41046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 -- 172.21.15.39:0/2394367151 >> 172.21.15.39:0/2394367151 conn(0x7f0ce40fbc30 msgr2=0x7f0ce40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 -- 172.21.15.39:0/2394367151 shutdown_connections 2024-03-16T10:06:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.978+0000 7f0ce9c00640 1 -- 172.21.15.39:0/2394367151 wait complete. 2024-03-16T10:06:42.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.979+0000 7f0ce9c00640 1 Processor -- start 2024-03-16T10:06:42.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.979+0000 7f0ce9c00640 1 -- start start 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 0x7f0ce419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410d4d0 con 0x7f0ce41022e0 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410d350 con 0x7f0ce4107540 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce9c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410d650 con 0x7f0ce4104c10 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48182/0 (socket says 172.21.15.39:48182) 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2a00640 1 -- 172.21.15.39:0/634198139 learned_addr learned my addr 172.21.15.39:0/634198139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 0x7f0ce419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce3400640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 -- 172.21.15.39:0/634198139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 msgr2=0x7f0ce41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 -- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 msgr2=0x7f0ce4199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2000640 1 -- 172.21.15.39:0/634198139 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce41a1900 con 0x7f0ce4104c10 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce2a00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.980+0000 7f0ce3400640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0ce2000640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 0x7f0ce419a200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000d6a0 tx=0x7f0cd000db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0017070 con 0x7f0ce4104c10 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd0009820 con 0x7f0ce4104c10 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0004d00 con 0x7f0ce4104c10 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce41a1bf0 con 0x7f0ce4104c10 2024-03-16T10:06:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.981+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a20f0 con 0x7f0ce4104c10 2024-03-16T10:06:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.982+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce4109030 con 0x7f0ce4104c10 2024-03-16T10:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.986+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0cd00099d0 con 0x7f0ce4104c10 2024-03-16T10:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.987+0000 7f0cdb400640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 0x7f0cc80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.987+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0cd00ce6e0 con 0x7f0ce4104c10 2024-03-16T10:06:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.987+0000 7f0ce2a00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 0x7f0cc80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.988+0000 7f0ce2a00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 0x7f0cc80a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ce419ab90 tx=0x7f0cd4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:42.993+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0cd009ad70 con 0x7f0ce4104c10 2024-03-16T10:06:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.193+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ce4061c20 con 0x7f0cc80a2890 2024-03-16T10:06:43.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.208+0000 7f0cdb400640 1 -- 172.21.15.39:0/634198139 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340650 (secure 0 0 0) 0x7f0ce4061c20 con 0x7f0cc80a2890 2024-03-16T10:06:43.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 msgr2=0x7f0cc80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 0x7f0cc80a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ce419ab90 tx=0x7f0cd4009500 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 msgr2=0x7f0ce419a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 0x7f0ce419a200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000d6a0 tx=0x7f0cd000db60 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 shutdown_connections 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0cc80a2890 0x7f0cc80a4d50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ce4107540 0x7f0ce41a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ce4104c10 0x7f0ce419a200 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 --2- 172.21.15.39:0/634198139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ce41022e0 0x7f0ce4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 >> 172.21.15.39:0/634198139 conn(0x7f0ce40fbc30 msgr2=0x7f0ce4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:43.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 shutdown_connections 2024-03-16T10:06:43.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.214+0000 7f0ce9c00640 1 -- 172.21.15.39:0/634198139 wait complete. 2024-03-16T10:06:43.234 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-03-16T10:06:42.213582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638498,"num_objects":95,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":168,"num_read_kb":185,"num_write":399,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352712,"kb_used_data":11876,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589112,"statfs":{"total":1151940427776,"available":1151579250688,"internally_reserved":0,"allocated":12161024,"data_stored":7614725,"data_compressed":285735,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43631,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":170,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931188+0000","last_change":"2024-03-16T10:06:04.221459+0000","last_active":"2024-03-16T10:06:09.931188+0000","last_peered":"2024-03-16T10:06:09.931188+0000","last_clean":"2024-03-16T10:06:09.931188+0000","last_became_active":"2024-03-16T10:06:04.221172+0000","last_became_peered":"2024-03-16T10:06:04.221172+0000","last_unstale":"2024-03-16T10:06:09.931188+0000","last_undegraded":"2024-03-16T10:06:09.931188+0000","last_fullsized":"2024-03-16T10:06:09.931188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:58:28.019985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337431+0000","last_change":"2024-03-16T10:06:04.205876+0000","last_active":"2024-03-16T10:06:09.337431+0000","last_peered":"2024-03-16T10:06:09.337431+0000","last_clean":"2024-03-16T10:06:09.337431+0000","last_became_active":"2024-03-16T10:06:04.205473+0000","last_became_peered":"2024-03-16T10:06:04.205473+0000","last_unstale":"2024-03-16T10:06:09.337431+0000","last_undegraded":"2024-03-16T10:06:09.337431+0000","last_fullsized":"2024-03-16T10:06:09.337431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:37.981148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336941+0000","last_change":"2024-03-16T10:06:04.197872+0000","last_active":"2024-03-16T10:06:09.336941+0000","last_peered":"2024-03-16T10:06:09.336941+0000","last_clean":"2024-03-16T10:06:09.336941+0000","last_became_active":"2024-03-16T10:06:04.197347+0000","last_became_peered":"2024-03-16T10:06:04.197347+0000","last_unstale":"2024-03-16T10:06:09.336941+0000","last_undegraded":"2024-03-16T10:06:09.336941+0000","last_fullsized":"2024-03-16T10:06:09.336941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:37.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929702+0000","last_change":"2024-03-16T10:06:04.196359+0000","last_active":"2024-03-16T10:06:09.929702+0000","last_peered":"2024-03-16T10:06:09.929702+0000","last_clean":"2024-03-16T10:06:09.929702+0000","last_became_active":"2024-03-16T10:06:04.196072+0000","last_became_peered":"2024-03-16T10:06:04.196072+0000","last_unstale":"2024-03-16T10:06:09.929702+0000","last_undegraded":"2024-03-16T10:06:09.929702+0000","last_fullsized":"2024-03-16T10:06:09.929702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:03.641536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930496+0000","last_change":"2024-03-16T10:06:04.224386+0000","last_active":"2024-03-16T10:06:09.930496+0000","last_peered":"2024-03-16T10:06:09.930496+0000","last_clean":"2024-03-16T10:06:09.930496+0000","last_became_active":"2024-03-16T10:06:04.224042+0000","last_became_peered":"2024-03-16T10:06:04.224042+0000","last_unstale":"2024-03-16T10:06:09.930496+0000","last_undegraded":"2024-03-16T10:06:09.930496+0000","last_fullsized":"2024-03-16T10:06:09.930496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:16:50.114249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930098+0000","last_change":"2024-03-16T10:06:04.214055+0000","last_active":"2024-03-16T10:06:09.930098+0000","last_peered":"2024-03-16T10:06:09.930098+0000","last_clean":"2024-03-16T10:06:09.930098+0000","last_became_active":"2024-03-16T10:06:04.213792+0000","last_became_peered":"2024-03-16T10:06:04.213792+0000","last_unstale":"2024-03-16T10:06:09.930098+0000","last_undegraded":"2024-03-16T10:06:09.930098+0000","last_fullsized":"2024-03-16T10:06:09.930098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:50.142741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599348+0000","last_change":"2024-03-16T10:06:04.208874+0000","last_active":"2024-03-16T10:06:09.599348+0000","last_peered":"2024-03-16T10:06:09.599348+0000","last_clean":"2024-03-16T10:06:09.599348+0000","last_became_active":"2024-03-16T10:06:04.208553+0000","last_became_peered":"2024-03-16T10:06:04.208553+0000","last_unstale":"2024-03-16T10:06:09.599348+0000","last_undegraded":"2024-03-16T10:06:09.599348+0000","last_fullsized":"2024-03-16T10:06:09.599348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:33.885089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415216+0000","last_change":"2024-03-16T10:06:04.233487+0000","last_active":"2024-03-16T10:06:09.415216+0000","last_peered":"2024-03-16T10:06:09.415216+0000","last_clean":"2024-03-16T10:06:09.415216+0000","last_became_active":"2024-03-16T10:06:04.228530+0000","last_became_peered":"2024-03-16T10:06:04.228530+0000","last_unstale":"2024-03-16T10:06:09.415216+0000","last_undegraded":"2024-03-16T10:06:09.415216+0000","last_fullsized":"2024-03-16T10:06:09.415216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:53:12.270229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333665+0000","last_change":"2024-03-16T10:06:04.190965+0000","last_active":"2024-03-16T10:06:09.333665+0000","last_peered":"2024-03-16T10:06:09.333665+0000","last_clean":"2024-03-16T10:06:09.333665+0000","last_became_active":"2024-03-16T10:06:04.190681+0000","last_became_peered":"2024-03-16T10:06:04.190681+0000","last_unstale":"2024-03-16T10:06:09.333665+0000","last_undegraded":"2024-03-16T10:06:09.333665+0000","last_fullsized":"2024-03-16T10:06:09.333665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:46.498680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930187+0000","last_change":"2024-03-16T10:06:04.198098+0000","last_active":"2024-03-16T10:06:09.930187+0000","last_peered":"2024-03-16T10:06:09.930187+0000","last_clean":"2024-03-16T10:06:09.930187+0000","last_became_active":"2024-03-16T10:06:04.197758+0000","last_became_peered":"2024-03-16T10:06:04.197758+0000","last_unstale":"2024-03-16T10:06:09.930187+0000","last_undegraded":"2024-03-16T10:06:09.930187+0000","last_fullsized":"2024-03-16T10:06:09.930187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:27.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599821+0000","last_change":"2024-03-16T10:06:04.195970+0000","last_active":"2024-03-16T10:06:09.599821+0000","last_peered":"2024-03-16T10:06:09.599821+0000","last_clean":"2024-03-16T10:06:09.599821+0000","last_became_active":"2024-03-16T10:06:04.195522+0000","last_became_peered":"2024-03-16T10:06:04.195522+0000","last_unstale":"2024-03-16T10:06:09.599821+0000","last_undegraded":"2024-03-16T10:06:09.599821+0000","last_fullsized":"2024-03-16T10:06:09.599821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:22.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929573+0000","last_change":"2024-03-16T10:06:04.196075+0000","last_active":"2024-03-16T10:06:09.929573+0000","last_peered":"2024-03-16T10:06:09.929573+0000","last_clean":"2024-03-16T10:06:09.929573+0000","last_became_active":"2024-03-16T10:06:04.195785+0000","last_became_peered":"2024-03-16T10:06:04.195785+0000","last_unstale":"2024-03-16T10:06:09.929573+0000","last_undegraded":"2024-03-16T10:06:09.929573+0000","last_fullsized":"2024-03-16T10:06:09.929573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:52:50.016787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932113+0000","last_change":"2024-03-16T10:06:04.206173+0000","last_active":"2024-03-16T10:06:09.932113+0000","last_peered":"2024-03-16T10:06:09.932113+0000","last_clean":"2024-03-16T10:06:09.932113+0000","last_became_active":"2024-03-16T10:06:04.205766+0000","last_became_peered":"2024-03-16T10:06:04.205766+0000","last_unstale":"2024-03-16T10:06:09.932113+0000","last_undegraded":"2024-03-16T10:06:09.932113+0000","last_fullsized":"2024-03-16T10:06:09.932113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:25:47.405044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336131+0000","last_change":"2024-03-16T10:06:04.196204+0000","last_active":"2024-03-16T10:06:09.336131+0000","last_peered":"2024-03-16T10:06:09.336131+0000","last_clean":"2024-03-16T10:06:09.336131+0000","last_became_active":"2024-03-16T10:06:04.194255+0000","last_became_peered":"2024-03-16T10:06:04.194255+0000","last_unstale":"2024-03-16T10:06:09.336131+0000","last_undegraded":"2024-03-16T10:06:09.336131+0000","last_fullsized":"2024-03-16T10:06:09.336131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:44.367876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.335954+0000","last_change":"2024-03-16T10:06:04.203298+0000","last_active":"2024-03-16T10:06:09.335954+0000","last_peered":"2024-03-16T10:06:09.335954+0000","last_clean":"2024-03-16T10:06:09.335954+0000","last_became_active":"2024-03-16T10:06:04.202790+0000","last_became_peered":"2024-03-16T10:06:04.202790+0000","last_unstale":"2024-03-16T10:06:09.335954+0000","last_undegraded":"2024-03-16T10:06:09.335954+0000","last_fullsized":"2024-03-16T10:06:09.335954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:27.360861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.332951+0000","last_change":"2024-03-16T10:06:04.191196+0000","last_active":"2024-03-16T10:06:09.332951+0000","last_peered":"2024-03-16T10:06:09.332951+0000","last_clean":"2024-03-16T10:06:09.332951+0000","last_became_active":"2024-03-16T10:06:04.190873+0000","last_became_peered":"2024-03-16T10:06:04.190873+0000","last_unstale":"2024-03-16T10:06:09.332951+0000","last_undegraded":"2024-03-16T10:06:09.332951+0000","last_fullsized":"2024-03-16T10:06:09.332951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:59.272388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930962+0000","last_change":"2024-03-16T10:06:04.221443+0000","last_active":"2024-03-16T10:06:09.930962+0000","last_peered":"2024-03-16T10:06:09.930962+0000","last_clean":"2024-03-16T10:06:09.930962+0000","last_became_active":"2024-03-16T10:06:04.221182+0000","last_became_peered":"2024-03-16T10:06:04.221182+0000","last_unstale":"2024-03-16T10:06:09.930962+0000","last_undegraded":"2024-03-16T10:06:09.930962+0000","last_fullsized":"2024-03-16T10:06:09.930962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:04:08.139784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931790+0000","last_change":"2024-03-16T10:06:04.208876+0000","last_active":"2024-03-16T10:06:09.931790+0000","last_peered":"2024-03-16T10:06:09.931790+0000","last_clean":"2024-03-16T10:06:09.931790+0000","last_became_active":"2024-03-16T10:06:04.208361+0000","last_became_peered":"2024-03-16T10:06:04.208361+0000","last_unstale":"2024-03-16T10:06:09.931790+0000","last_undegraded":"2024-03-16T10:06:09.931790+0000","last_fullsized":"2024-03-16T10:06:09.931790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:53.156832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931108+0000","last_change":"2024-03-16T10:06:04.228080+0000","last_active":"2024-03-16T10:06:09.931108+0000","last_peered":"2024-03-16T10:06:09.931108+0000","last_clean":"2024-03-16T10:06:09.931108+0000","last_became_active":"2024-03-16T10:06:04.227799+0000","last_became_peered":"2024-03-16T10:06:04.227799+0000","last_unstale":"2024-03-16T10:06:09.931108+0000","last_undegraded":"2024-03-16T10:06:09.931108+0000","last_fullsized":"2024-03-16T10:06:09.931108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:40:55.113354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930478+0000","last_change":"2024-03-16T10:06:08.349744+0000","last_active":"2024-03-16T10:06:09.930478+0000","last_peered":"2024-03-16T10:06:09.930478+0000","last_clean":"2024-03-16T10:06:09.930478+0000","last_became_active":"2024-03-16T10:06:08.349359+0000","last_became_peered":"2024-03-16T10:06:08.349359+0000","last_unstale":"2024-03-16T10:06:09.930478+0000","last_undegraded":"2024-03-16T10:06:09.930478+0000","last_fullsized":"2024-03-16T10:06:09.930478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:30:02.614189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.600329+0000","last_change":"2024-03-16T10:06:03.234836+0000","last_active":"2024-03-16T10:06:40.600329+0000","last_peered":"2024-03-16T10:06:40.600329+0000","last_clean":"2024-03-16T10:06:40.600329+0000","last_became_active":"2024-03-16T10:06:03.234436+0000","last_became_peered":"2024-03-16T10:06:03.234436+0000","last_unstale":"2024-03-16T10:06:40.600329+0000","last_undegraded":"2024-03-16T10:06:40.600329+0000","last_fullsized":"2024-03-16T10:06:40.600329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:52:34.069875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416167+0000","last_change":"2024-03-16T10:06:04.222012+0000","last_active":"2024-03-16T10:06:09.416167+0000","last_peered":"2024-03-16T10:06:09.416167+0000","last_clean":"2024-03-16T10:06:09.416167+0000","last_became_active":"2024-03-16T10:06:04.221661+0000","last_became_peered":"2024-03-16T10:06:04.221661+0000","last_unstale":"2024-03-16T10:06:09.416167+0000","last_undegraded":"2024-03-16T10:06:09.416167+0000","last_fullsized":"2024-03-16T10:06:09.416167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:07.837752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929109+0000","last_change":"2024-03-16T10:06:08.345658+0000","last_active":"2024-03-16T10:06:09.929109+0000","last_peered":"2024-03-16T10:06:09.929109+0000","last_clean":"2024-03-16T10:06:09.929109+0000","last_became_active":"2024-03-16T10:06:08.345291+0000","last_became_peered":"2024-03-16T10:06:08.345291+0000","last_unstale":"2024-03-16T10:06:09.929109+0000","last_undegraded":"2024-03-16T10:06:09.929109+0000","last_fullsized":"2024-03-16T10:06:09.929109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:43:39.734164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416279+0000","last_change":"2024-03-16T10:06:03.244368+0000","last_active":"2024-03-16T10:06:09.416279+0000","last_peered":"2024-03-16T10:06:09.416279+0000","last_clean":"2024-03-16T10:06:09.416279+0000","last_became_active":"2024-03-16T10:06:03.243369+0000","last_became_peered":"2024-03-16T10:06:03.243369+0000","last_unstale":"2024-03-16T10:06:09.416279+0000","last_undegraded":"2024-03-16T10:06:09.416279+0000","last_fullsized":"2024-03-16T10:06:09.416279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:38.473988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337432+0000","last_change":"2024-03-16T10:06:04.217255+0000","last_active":"2024-03-16T10:06:09.337432+0000","last_peered":"2024-03-16T10:06:09.337432+0000","last_clean":"2024-03-16T10:06:09.337432+0000","last_became_active":"2024-03-16T10:06:04.216659+0000","last_became_peered":"2024-03-16T10:06:04.216659+0000","last_unstale":"2024-03-16T10:06:09.337432+0000","last_undegraded":"2024-03-16T10:06:09.337432+0000","last_fullsized":"2024-03-16T10:06:09.337432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:06.081507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929293+0000","last_change":"2024-03-16T10:06:08.346622+0000","last_active":"2024-03-16T10:06:09.929293+0000","last_peered":"2024-03-16T10:06:09.929293+0000","last_clean":"2024-03-16T10:06:09.929293+0000","last_became_active":"2024-03-16T10:06:08.346209+0000","last_became_peered":"2024-03-16T10:06:08.346209+0000","last_unstale":"2024-03-16T10:06:09.929293+0000","last_undegraded":"2024-03-16T10:06:09.929293+0000","last_fullsized":"2024-03-16T10:06:09.929293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:46:07.305218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.598761+0000","last_change":"2024-03-16T10:06:03.227303+0000","last_active":"2024-03-16T10:06:40.598761+0000","last_peered":"2024-03-16T10:06:40.598761+0000","last_clean":"2024-03-16T10:06:40.598761+0000","last_became_active":"2024-03-16T10:06:03.226953+0000","last_became_peered":"2024-03-16T10:06:03.226953+0000","last_unstale":"2024-03-16T10:06:40.598761+0000","last_undegraded":"2024-03-16T10:06:40.598761+0000","last_fullsized":"2024-03-16T10:06:40.598761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:28:40.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333170+0000","last_change":"2024-03-16T10:06:04.220502+0000","last_active":"2024-03-16T10:06:09.333170+0000","last_peered":"2024-03-16T10:06:09.333170+0000","last_clean":"2024-03-16T10:06:09.333170+0000","last_became_active":"2024-03-16T10:06:04.220057+0000","last_became_peered":"2024-03-16T10:06:04.220057+0000","last_unstale":"2024-03-16T10:06:09.333170+0000","last_undegraded":"2024-03-16T10:06:09.333170+0000","last_fullsized":"2024-03-16T10:06:09.333170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:46:30.972382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929131+0000","last_change":"2024-03-16T10:06:08.346548+0000","last_active":"2024-03-16T10:06:09.929131+0000","last_peered":"2024-03-16T10:06:09.929131+0000","last_clean":"2024-03-16T10:06:09.929131+0000","last_became_active":"2024-03-16T10:06:08.346145+0000","last_became_peered":"2024-03-16T10:06:08.346145+0000","last_unstale":"2024-03-16T10:06:09.929131+0000","last_undegraded":"2024-03-16T10:06:09.929131+0000","last_fullsized":"2024-03-16T10:06:09.929131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:49.221773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.569909+0000","last_change":"2024-03-16T10:06:03.222623+0000","last_active":"2024-03-16T10:06:39.569909+0000","last_peered":"2024-03-16T10:06:39.569909+0000","last_clean":"2024-03-16T10:06:39.569909+0000","last_became_active":"2024-03-16T10:06:03.222300+0000","last_became_peered":"2024-03-16T10:06:03.222300+0000","last_unstale":"2024-03-16T10:06:39.569909+0000","last_undegraded":"2024-03-16T10:06:39.569909+0000","last_fullsized":"2024-03-16T10:06:39.569909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:49.397327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333607+0000","last_change":"2024-03-16T10:06:04.194004+0000","last_active":"2024-03-16T10:06:09.333607+0000","last_peered":"2024-03-16T10:06:09.333607+0000","last_clean":"2024-03-16T10:06:09.333607+0000","last_became_active":"2024-03-16T10:06:04.193551+0000","last_became_peered":"2024-03-16T10:06:04.193551+0000","last_unstale":"2024-03-16T10:06:09.333607+0000","last_undegraded":"2024-03-16T10:06:09.333607+0000","last_fullsized":"2024-03-16T10:06:09.333607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:59:25.668511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930255+0000","last_change":"2024-03-16T10:06:08.330170+0000","last_active":"2024-03-16T10:06:09.930255+0000","last_peered":"2024-03-16T10:06:09.930255+0000","last_clean":"2024-03-16T10:06:09.930255+0000","last_became_active":"2024-03-16T10:06:08.329699+0000","last_became_peered":"2024-03-16T10:06:08.329699+0000","last_unstale":"2024-03-16T10:06:09.930255+0000","last_undegraded":"2024-03-16T10:06:09.930255+0000","last_fullsized":"2024-03-16T10:06:09.930255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:50.571845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.518523+0000","last_change":"2024-03-16T10:06:03.228039+0000","last_active":"2024-03-16T10:06:36.518523+0000","last_peered":"2024-03-16T10:06:36.518523+0000","last_clean":"2024-03-16T10:06:36.518523+0000","last_became_active":"2024-03-16T10:06:03.227715+0000","last_became_peered":"2024-03-16T10:06:03.227715+0000","last_unstale":"2024-03-16T10:06:36.518523+0000","last_undegraded":"2024-03-16T10:06:36.518523+0000","last_fullsized":"2024-03-16T10:06:36.518523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:13:08.980781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931051+0000","last_change":"2024-03-16T10:06:04.214274+0000","last_active":"2024-03-16T10:06:09.931051+0000","last_peered":"2024-03-16T10:06:09.931051+0000","last_clean":"2024-03-16T10:06:09.931051+0000","last_became_active":"2024-03-16T10:06:04.213979+0000","last_became_peered":"2024-03-16T10:06:04.213979+0000","last_unstale":"2024-03-16T10:06:09.931051+0000","last_undegraded":"2024-03-16T10:06:09.931051+0000","last_fullsized":"2024-03-16T10:06:09.931051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:12.684609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930021+0000","last_change":"2024-03-16T10:06:08.346874+0000","last_active":"2024-03-16T10:06:09.930021+0000","last_peered":"2024-03-16T10:06:09.930021+0000","last_clean":"2024-03-16T10:06:09.930021+0000","last_became_active":"2024-03-16T10:06:08.346450+0000","last_became_peered":"2024-03-16T10:06:08.346450+0000","last_unstale":"2024-03-16T10:06:09.930021+0000","last_undegraded":"2024-03-16T10:06:09.930021+0000","last_fullsized":"2024-03-16T10:06:09.930021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:09:12.159483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.556116+0000","last_change":"2024-03-16T10:06:03.236088+0000","last_active":"2024-03-16T10:06:38.556116+0000","last_peered":"2024-03-16T10:06:38.556116+0000","last_clean":"2024-03-16T10:06:38.556116+0000","last_became_active":"2024-03-16T10:06:03.235768+0000","last_became_peered":"2024-03-16T10:06:03.235768+0000","last_unstale":"2024-03-16T10:06:38.556116+0000","last_undegraded":"2024-03-16T10:06:38.556116+0000","last_fullsized":"2024-03-16T10:06:38.556116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:57:38.813913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333976+0000","last_change":"2024-03-16T10:06:04.224331+0000","last_active":"2024-03-16T10:06:09.333976+0000","last_peered":"2024-03-16T10:06:09.333976+0000","last_clean":"2024-03-16T10:06:09.333976+0000","last_became_active":"2024-03-16T10:06:04.223946+0000","last_became_peered":"2024-03-16T10:06:04.223946+0000","last_unstale":"2024-03-16T10:06:09.333976+0000","last_undegraded":"2024-03-16T10:06:09.333976+0000","last_fullsized":"2024-03-16T10:06:09.333976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:58:56.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930996+0000","last_change":"2024-03-16T10:06:08.344757+0000","last_active":"2024-03-16T10:06:09.930996+0000","last_peered":"2024-03-16T10:06:09.930996+0000","last_clean":"2024-03-16T10:06:09.930996+0000","last_became_active":"2024-03-16T10:06:08.344275+0000","last_became_peered":"2024-03-16T10:06:08.344275+0000","last_unstale":"2024-03-16T10:06:09.930996+0000","last_undegraded":"2024-03-16T10:06:09.930996+0000","last_fullsized":"2024-03-16T10:06:09.930996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:31:18.545734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:37.539735+0000","last_change":"2024-03-16T10:06:03.239619+0000","last_active":"2024-03-16T10:06:37.539735+0000","last_peered":"2024-03-16T10:06:37.539735+0000","last_clean":"2024-03-16T10:06:37.539735+0000","last_became_active":"2024-03-16T10:06:03.238982+0000","last_became_peered":"2024-03-16T10:06:03.238982+0000","last_unstale":"2024-03-16T10:06:37.539735+0000","last_undegraded":"2024-03-16T10:06:37.539735+0000","last_fullsized":"2024-03-16T10:06:37.539735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:20.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415080+0000","last_change":"2024-03-16T10:06:04.221897+0000","last_active":"2024-03-16T10:06:09.415080+0000","last_peered":"2024-03-16T10:06:09.415080+0000","last_clean":"2024-03-16T10:06:09.415080+0000","last_became_active":"2024-03-16T10:06:04.218243+0000","last_became_peered":"2024-03-16T10:06:04.218243+0000","last_unstale":"2024-03-16T10:06:09.415080+0000","last_undegraded":"2024-03-16T10:06:09.415080+0000","last_fullsized":"2024-03-16T10:06:09.415080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:25.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337260+0000","last_change":"2024-03-16T10:06:08.352722+0000","last_active":"2024-03-16T10:06:09.337260+0000","last_peered":"2024-03-16T10:06:09.337260+0000","last_clean":"2024-03-16T10:06:09.337260+0000","last_became_active":"2024-03-16T10:06:08.352292+0000","last_became_peered":"2024-03-16T10:06:08.352292+0000","last_unstale":"2024-03-16T10:06:09.337260+0000","last_undegraded":"2024-03-16T10:06:09.337260+0000","last_fullsized":"2024-03-16T10:06:09.337260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:56:23.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333255+0000","last_change":"2024-03-16T10:06:03.224143+0000","last_active":"2024-03-16T10:06:09.333255+0000","last_peered":"2024-03-16T10:06:09.333255+0000","last_clean":"2024-03-16T10:06:09.333255+0000","last_became_active":"2024-03-16T10:06:03.223810+0000","last_became_peered":"2024-03-16T10:06:03.223810+0000","last_unstale":"2024-03-16T10:06:09.333255+0000","last_undegraded":"2024-03-16T10:06:09.333255+0000","last_fullsized":"2024-03-16T10:06:09.333255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:26.724292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930872+0000","last_change":"2024-03-16T10:06:04.232758+0000","last_active":"2024-03-16T10:06:09.930872+0000","last_peered":"2024-03-16T10:06:09.930872+0000","last_clean":"2024-03-16T10:06:09.930872+0000","last_became_active":"2024-03-16T10:06:04.232371+0000","last_became_peered":"2024-03-16T10:06:04.232371+0000","last_unstale":"2024-03-16T10:06:09.930872+0000","last_undegraded":"2024-03-16T10:06:09.930872+0000","last_fullsized":"2024-03-16T10:06:09.930872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:00.527708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415720+0000","last_change":"2024-03-16T10:06:08.345360+0000","last_active":"2024-03-16T10:06:09.415720+0000","last_peered":"2024-03-16T10:06:09.415720+0000","last_clean":"2024-03-16T10:06:09.415720+0000","last_became_active":"2024-03-16T10:06:08.336245+0000","last_became_peered":"2024-03-16T10:06:08.336245+0000","last_unstale":"2024-03-16T10:06:09.415720+0000","last_undegraded":"2024-03-16T10:06:09.415720+0000","last_fullsized":"2024-03-16T10:06:09.415720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:14:33.324555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:41.600709+0000","last_change":"2024-03-16T10:06:03.237015+0000","last_active":"2024-03-16T10:06:41.600709+0000","last_peered":"2024-03-16T10:06:41.600709+0000","last_clean":"2024-03-16T10:06:41.600709+0000","last_became_active":"2024-03-16T10:06:03.236722+0000","last_became_peered":"2024-03-16T10:06:03.236722+0000","last_unstale":"2024-03-16T10:06:41.600709+0000","last_undegraded":"2024-03-16T10:06:41.600709+0000","last_fullsized":"2024-03-16T10:06:41.600709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:10:03.403854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333627+0000","last_change":"2024-03-16T10:06:04.209017+0000","last_active":"2024-03-16T10:06:09.333627+0000","last_peered":"2024-03-16T10:06:09.333627+0000","last_clean":"2024-03-16T10:06:09.333627+0000","last_became_active":"2024-03-16T10:06:04.208665+0000","last_became_peered":"2024-03-16T10:06:04.208665+0000","last_unstale":"2024-03-16T10:06:09.333627+0000","last_undegraded":"2024-03-16T10:06:09.333627+0000","last_fullsized":"2024-03-16T10:06:09.333627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:27.074480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931172+0000","last_change":"2024-03-16T10:06:08.340510+0000","last_active":"2024-03-16T10:06:09.931172+0000","last_peered":"2024-03-16T10:06:09.931172+0000","last_clean":"2024-03-16T10:06:09.931172+0000","last_became_active":"2024-03-16T10:06:08.340070+0000","last_became_peered":"2024-03-16T10:06:08.340070+0000","last_unstale":"2024-03-16T10:06:09.931172+0000","last_undegraded":"2024-03-16T10:06:09.931172+0000","last_fullsized":"2024-03-16T10:06:09.931172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:33:55.038855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.569218+0000","last_change":"2024-03-16T10:06:03.225770+0000","last_active":"2024-03-16T10:06:39.569218+0000","last_peered":"2024-03-16T10:06:39.569218+0000","last_clean":"2024-03-16T10:06:39.569218+0000","last_became_active":"2024-03-16T10:06:03.223734+0000","last_became_peered":"2024-03-16T10:06:03.223734+0000","last_unstale":"2024-03-16T10:06:39.569218+0000","last_undegraded":"2024-03-16T10:06:39.569218+0000","last_fullsized":"2024-03-16T10:06:39.569218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:13.764147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337450+0000","last_change":"2024-03-16T10:06:04.206027+0000","last_active":"2024-03-16T10:06:09.337450+0000","last_peered":"2024-03-16T10:06:09.337450+0000","last_clean":"2024-03-16T10:06:09.337450+0000","last_became_active":"2024-03-16T10:06:04.191657+0000","last_became_peered":"2024-03-16T10:06:04.191657+0000","last_unstale":"2024-03-16T10:06:09.337450+0000","last_undegraded":"2024-03-16T10:06:09.337450+0000","last_fullsized":"2024-03-16T10:06:09.337450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:52.839646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929615+0000","last_change":"2024-03-16T10:06:08.345595+0000","last_active":"2024-03-16T10:06:09.929615+0000","last_peered":"2024-03-16T10:06:09.929615+0000","last_clean":"2024-03-16T10:06:09.929615+0000","last_became_active":"2024-03-16T10:06:08.345205+0000","last_became_peered":"2024-03-16T10:06:08.345205+0000","last_unstale":"2024-03-16T10:06:09.929615+0000","last_undegraded":"2024-03-16T10:06:09.929615+0000","last_fullsized":"2024-03-16T10:06:09.929615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:37:51.067056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333485+0000","last_change":"2024-03-16T10:06:03.219716+0000","last_active":"2024-03-16T10:06:09.333485+0000","last_peered":"2024-03-16T10:06:09.333485+0000","last_clean":"2024-03-16T10:06:09.333485+0000","last_became_active":"2024-03-16T10:06:03.219375+0000","last_became_peered":"2024-03-16T10:06:03.219375+0000","last_unstale":"2024-03-16T10:06:09.333485+0000","last_undegraded":"2024-03-16T10:06:09.333485+0000","last_fullsized":"2024-03-16T10:06:09.333485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:23:51.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333378+0000","last_change":"2024-03-16T10:06:04.193605+0000","last_active":"2024-03-16T10:06:09.333378+0000","last_peered":"2024-03-16T10:06:09.333378+0000","last_clean":"2024-03-16T10:06:09.333378+0000","last_became_active":"2024-03-16T10:06:04.193295+0000","last_became_peered":"2024-03-16T10:06:04.193295+0000","last_unstale":"2024-03-16T10:06:09.333378+0000","last_undegraded":"2024-03-16T10:06:09.333378+0000","last_fullsized":"2024-03-16T10:06:09.333378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:10:04.140279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931302+0000","last_change":"2024-03-16T10:06:08.338862+0000","last_active":"2024-03-16T10:06:09.931302+0000","last_peered":"2024-03-16T10:06:09.931302+0000","last_clean":"2024-03-16T10:06:09.931302+0000","last_became_active":"2024-03-16T10:06:08.337221+0000","last_became_peered":"2024-03-16T10:06:08.337221+0000","last_unstale":"2024-03-16T10:06:09.931302+0000","last_undegraded":"2024-03-16T10:06:09.931302+0000","last_fullsized":"2024-03-16T10:06:09.931302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:28:51.251299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:41.606270+0000","last_change":"2024-03-16T10:06:03.220555+0000","last_active":"2024-03-16T10:06:41.606270+0000","last_peered":"2024-03-16T10:06:41.606270+0000","last_clean":"2024-03-16T10:06:41.606270+0000","last_became_active":"2024-03-16T10:06:03.220224+0000","last_became_peered":"2024-03-16T10:06:03.220224+0000","last_unstale":"2024-03-16T10:06:41.606270+0000","last_undegraded":"2024-03-16T10:06:41.606270+0000","last_fullsized":"2024-03-16T10:06:41.606270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:12:47.568132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931288+0000","last_change":"2024-03-16T10:06:04.195957+0000","last_active":"2024-03-16T10:06:09.931288+0000","last_peered":"2024-03-16T10:06:09.931288+0000","last_clean":"2024-03-16T10:06:09.931288+0000","last_became_active":"2024-03-16T10:06:04.195539+0000","last_became_peered":"2024-03-16T10:06:04.195539+0000","last_unstale":"2024-03-16T10:06:09.931288+0000","last_undegraded":"2024-03-16T10:06:09.931288+0000","last_fullsized":"2024-03-16T10:06:09.931288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:39:45.035936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.334156+0000","last_change":"2024-03-16T10:06:08.347941+0000","last_active":"2024-03-16T10:06:09.334156+0000","last_peered":"2024-03-16T10:06:09.334156+0000","last_clean":"2024-03-16T10:06:09.334156+0000","last_became_active":"2024-03-16T10:06:08.347517+0000","last_became_peered":"2024-03-16T10:06:08.347517+0000","last_unstale":"2024-03-16T10:06:09.334156+0000","last_undegraded":"2024-03-16T10:06:09.334156+0000","last_fullsized":"2024-03-16T10:06:09.334156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:51:08.696206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.598434+0000","last_change":"2024-03-16T10:06:03.235652+0000","last_active":"2024-03-16T10:06:40.598434+0000","last_peered":"2024-03-16T10:06:40.598434+0000","last_clean":"2024-03-16T10:06:40.598434+0000","last_became_active":"2024-03-16T10:06:03.235219+0000","last_became_peered":"2024-03-16T10:06:03.235219+0000","last_unstale":"2024-03-16T10:06:40.598434+0000","last_undegraded":"2024-03-16T10:06:40.598434+0000","last_fullsized":"2024-03-16T10:06:40.598434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:23.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931581+0000","last_change":"2024-03-16T10:06:04.225460+0000","last_active":"2024-03-16T10:06:09.931581+0000","last_peered":"2024-03-16T10:06:09.931581+0000","last_clean":"2024-03-16T10:06:09.931581+0000","last_became_active":"2024-03-16T10:06:04.225019+0000","last_became_peered":"2024-03-16T10:06:04.225019+0000","last_unstale":"2024-03-16T10:06:09.931581+0000","last_undegraded":"2024-03-16T10:06:09.931581+0000","last_fullsized":"2024-03-16T10:06:09.931581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:42:06.525298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930960+0000","last_change":"2024-03-16T10:06:08.344775+0000","last_active":"2024-03-16T10:06:09.930960+0000","last_peered":"2024-03-16T10:06:09.930960+0000","last_clean":"2024-03-16T10:06:09.930960+0000","last_became_active":"2024-03-16T10:06:08.344360+0000","last_became_peered":"2024-03-16T10:06:08.344360+0000","last_unstale":"2024-03-16T10:06:09.930960+0000","last_undegraded":"2024-03-16T10:06:09.930960+0000","last_fullsized":"2024-03-16T10:06:09.930960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:26.710955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.519873+0000","last_change":"2024-03-16T10:06:03.236887+0000","last_active":"2024-03-16T10:06:36.519873+0000","last_peered":"2024-03-16T10:06:36.519873+0000","last_clean":"2024-03-16T10:06:36.519873+0000","last_became_active":"2024-03-16T10:06:03.236454+0000","last_became_peered":"2024-03-16T10:06:03.236454+0000","last_unstale":"2024-03-16T10:06:36.519873+0000","last_undegraded":"2024-03-16T10:06:36.519873+0000","last_fullsized":"2024-03-16T10:06:36.519873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:03.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599575+0000","last_change":"2024-03-16T10:06:04.220008+0000","last_active":"2024-03-16T10:06:09.599575+0000","last_peered":"2024-03-16T10:06:09.599575+0000","last_clean":"2024-03-16T10:06:09.599575+0000","last_became_active":"2024-03-16T10:06:04.219683+0000","last_became_peered":"2024-03-16T10:06:04.219683+0000","last_unstale":"2024-03-16T10:06:09.599575+0000","last_undegraded":"2024-03-16T10:06:09.599575+0000","last_fullsized":"2024-03-16T10:06:09.599575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:35.577474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931338+0000","last_change":"2024-03-16T10:06:08.352793+0000","last_active":"2024-03-16T10:06:09.931338+0000","last_peered":"2024-03-16T10:06:09.931338+0000","last_clean":"2024-03-16T10:06:09.931338+0000","last_became_active":"2024-03-16T10:06:08.352272+0000","last_became_peered":"2024-03-16T10:06:08.352272+0000","last_unstale":"2024-03-16T10:06:09.931338+0000","last_undegraded":"2024-03-16T10:06:09.931338+0000","last_fullsized":"2024-03-16T10:06:09.931338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:39.835078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.550722+0000","last_change":"2024-03-16T10:06:03.224301+0000","last_active":"2024-03-16T10:06:38.550722+0000","last_peered":"2024-03-16T10:06:38.550722+0000","last_clean":"2024-03-16T10:06:38.550722+0000","last_became_active":"2024-03-16T10:06:03.223899+0000","last_became_peered":"2024-03-16T10:06:03.223899+0000","last_unstale":"2024-03-16T10:06:38.550722+0000","last_undegraded":"2024-03-16T10:06:38.550722+0000","last_fullsized":"2024-03-16T10:06:38.550722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:39.244106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599346+0000","last_change":"2024-03-16T10:06:04.193649+0000","last_active":"2024-03-16T10:06:09.599346+0000","last_peered":"2024-03-16T10:06:09.599346+0000","last_clean":"2024-03-16T10:06:09.599346+0000","last_became_active":"2024-03-16T10:06:04.193275+0000","last_became_peered":"2024-03-16T10:06:04.193275+0000","last_unstale":"2024-03-16T10:06:09.599346+0000","last_undegraded":"2024-03-16T10:06:09.599346+0000","last_fullsized":"2024-03-16T10:06:09.599346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:59.597769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932892+0000","last_change":"2024-03-16T10:06:08.349918+0000","last_active":"2024-03-16T10:06:09.932892+0000","last_peered":"2024-03-16T10:06:09.932892+0000","last_clean":"2024-03-16T10:06:09.932892+0000","last_became_active":"2024-03-16T10:06:08.349506+0000","last_became_peered":"2024-03-16T10:06:08.349506+0000","last_unstale":"2024-03-16T10:06:09.932892+0000","last_undegraded":"2024-03-16T10:06:09.932892+0000","last_fullsized":"2024-03-16T10:06:09.932892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:08.146594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.568645+0000","last_change":"2024-03-16T10:06:03.231888+0000","last_active":"2024-03-16T10:06:39.568645+0000","last_peered":"2024-03-16T10:06:39.568645+0000","last_clean":"2024-03-16T10:06:39.568645+0000","last_became_active":"2024-03-16T10:06:03.231576+0000","last_became_peered":"2024-03-16T10:06:03.231576+0000","last_unstale":"2024-03-16T10:06:39.568645+0000","last_undegraded":"2024-03-16T10:06:39.568645+0000","last_fullsized":"2024-03-16T10:06:39.568645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:46:48.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333183+0000","last_change":"2024-03-16T10:06:04.194629+0000","last_active":"2024-03-16T10:06:09.333183+0000","last_peered":"2024-03-16T10:06:09.333183+0000","last_clean":"2024-03-16T10:06:09.333183+0000","last_became_active":"2024-03-16T10:06:04.194306+0000","last_became_peered":"2024-03-16T10:06:04.194306+0000","last_unstale":"2024-03-16T10:06:09.333183+0000","last_undegraded":"2024-03-16T10:06:09.333183+0000","last_fullsized":"2024-03-16T10:06:09.333183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:36.455569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333244+0000","last_change":"2024-03-16T10:06:08.349269+0000","last_active":"2024-03-16T10:06:09.333244+0000","last_peered":"2024-03-16T10:06:09.333244+0000","last_clean":"2024-03-16T10:06:09.333244+0000","last_became_active":"2024-03-16T10:06:08.348824+0000","last_became_peered":"2024-03-16T10:06:08.348824+0000","last_unstale":"2024-03-16T10:06:09.333244+0000","last_undegraded":"2024-03-16T10:06:09.333244+0000","last_fullsized":"2024-03-16T10:06:09.333244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:26.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.532429+0000","last_change":"2024-03-16T10:06:03.221066+0000","last_active":"2024-03-16T10:06:39.532429+0000","last_peered":"2024-03-16T10:06:39.532429+0000","last_clean":"2024-03-16T10:06:39.532429+0000","last_became_active":"2024-03-16T10:06:03.219908+0000","last_became_peered":"2024-03-16T10:06:03.219908+0000","last_unstale":"2024-03-16T10:06:39.532429+0000","last_undegraded":"2024-03-16T10:06:39.532429+0000","last_fullsized":"2024-03-16T10:06:39.532429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:41.026293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930634+0000","last_change":"2024-03-16T10:06:04.198752+0000","last_active":"2024-03-16T10:06:09.930634+0000","last_peered":"2024-03-16T10:06:09.930634+0000","last_clean":"2024-03-16T10:06:09.930634+0000","last_became_active":"2024-03-16T10:06:04.198016+0000","last_became_peered":"2024-03-16T10:06:04.198016+0000","last_unstale":"2024-03-16T10:06:09.930634+0000","last_undegraded":"2024-03-16T10:06:09.930634+0000","last_fullsized":"2024-03-16T10:06:09.930634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:09.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930986+0000","last_change":"2024-03-16T10:06:08.529441+0000","last_active":"2024-03-16T10:06:09.930986+0000","last_peered":"2024-03-16T10:06:09.930986+0000","last_clean":"2024-03-16T10:06:09.930986+0000","last_became_active":"2024-03-16T10:06:08.528801+0000","last_became_peered":"2024-03-16T10:06:08.528801+0000","last_unstale":"2024-03-16T10:06:09.930986+0000","last_undegraded":"2024-03-16T10:06:09.930986+0000","last_fullsized":"2024-03-16T10:06:09.930986+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:50.592415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931113+0000","last_change":"2024-03-16T10:06:03.236260+0000","last_active":"2024-03-16T10:06:09.931113+0000","last_peered":"2024-03-16T10:06:09.931113+0000","last_clean":"2024-03-16T10:06:09.931113+0000","last_became_active":"2024-03-16T10:06:03.235792+0000","last_became_peered":"2024-03-16T10:06:03.235792+0000","last_unstale":"2024-03-16T10:06:09.931113+0000","last_undegraded":"2024-03-16T10:06:09.931113+0000","last_fullsized":"2024-03-16T10:06:09.931113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:11:59.364244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930244+0000","last_change":"2024-03-16T10:06:04.226962+0000","last_active":"2024-03-16T10:06:09.930244+0000","last_peered":"2024-03-16T10:06:09.930244+0000","last_clean":"2024-03-16T10:06:09.930244+0000","last_became_active":"2024-03-16T10:06:04.226668+0000","last_became_peered":"2024-03-16T10:06:04.226668+0000","last_unstale":"2024-03-16T10:06:09.930244+0000","last_undegraded":"2024-03-16T10:06:09.930244+0000","last_fullsized":"2024-03-16T10:06:09.930244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:53.097834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932114+0000","last_change":"2024-03-16T10:06:08.343528+0000","last_active":"2024-03-16T10:06:09.932114+0000","last_peered":"2024-03-16T10:06:09.932114+0000","last_clean":"2024-03-16T10:06:09.932114+0000","last_became_active":"2024-03-16T10:06:08.340969+0000","last_became_peered":"2024-03-16T10:06:08.340969+0000","last_unstale":"2024-03-16T10:06:09.932114+0000","last_undegraded":"2024-03-16T10:06:09.932114+0000","last_fullsized":"2024-03-16T10:06:09.932114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:54:33.028756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.602835+0000","last_change":"2024-03-16T10:06:03.236166+0000","last_active":"2024-03-16T10:06:40.602835+0000","last_peered":"2024-03-16T10:06:40.602835+0000","last_clean":"2024-03-16T10:06:40.602835+0000","last_became_active":"2024-03-16T10:06:03.235825+0000","last_became_peered":"2024-03-16T10:06:03.235825+0000","last_unstale":"2024-03-16T10:06:40.602835+0000","last_undegraded":"2024-03-16T10:06:40.602835+0000","last_fullsized":"2024-03-16T10:06:40.602835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:39:34.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929916+0000","last_change":"2024-03-16T10:06:04.220002+0000","last_active":"2024-03-16T10:06:09.929916+0000","last_peered":"2024-03-16T10:06:09.929916+0000","last_clean":"2024-03-16T10:06:09.929916+0000","last_became_active":"2024-03-16T10:06:04.219623+0000","last_became_peered":"2024-03-16T10:06:04.219623+0000","last_unstale":"2024-03-16T10:06:09.929916+0000","last_undegraded":"2024-03-16T10:06:09.929916+0000","last_fullsized":"2024-03-16T10:06:09.929916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:39:35.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932650+0000","last_change":"2024-03-16T10:06:08.530396+0000","last_active":"2024-03-16T10:06:09.932650+0000","last_peered":"2024-03-16T10:06:09.932650+0000","last_clean":"2024-03-16T10:06:09.932650+0000","last_became_active":"2024-03-16T10:06:08.529102+0000","last_became_peered":"2024-03-16T10:06:08.529102+0000","last_unstale":"2024-03-16T10:06:09.932650+0000","last_undegraded":"2024-03-16T10:06:09.932650+0000","last_fullsized":"2024-03-16T10:06:09.932650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:36.408884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333576+0000","last_change":"2024-03-16T10:06:04.193150+0000","last_active":"2024-03-16T10:06:09.333576+0000","last_peered":"2024-03-16T10:06:09.333576+0000","last_clean":"2024-03-16T10:06:09.333576+0000","last_became_active":"2024-03-16T10:06:04.192847+0000","last_became_peered":"2024-03-16T10:06:04.192847+0000","last_unstale":"2024-03-16T10:06:09.333576+0000","last_undegraded":"2024-03-16T10:06:09.333576+0000","last_fullsized":"2024-03-16T10:06:09.333576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:31:22.687107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.600754+0000","last_change":"2024-03-16T10:06:03.247109+0000","last_active":"2024-03-16T10:06:40.600754+0000","last_peered":"2024-03-16T10:06:40.600754+0000","last_clean":"2024-03-16T10:06:40.600754+0000","last_became_active":"2024-03-16T10:06:03.246515+0000","last_became_peered":"2024-03-16T10:06:03.246515+0000","last_unstale":"2024-03-16T10:06:40.600754+0000","last_undegraded":"2024-03-16T10:06:40.600754+0000","last_fullsized":"2024-03-16T10:06:40.600754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:29.744828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:11.723238+0000","last_change":"2024-03-16T10:06:08.334587+0000","last_active":"2024-03-16T10:06:11.723238+0000","last_peered":"2024-03-16T10:06:11.723238+0000","last_clean":"2024-03-16T10:06:11.723238+0000","last_became_active":"2024-03-16T10:06:08.334248+0000","last_became_peered":"2024-03-16T10:06:08.334248+0000","last_unstale":"2024-03-16T10:06:11.723238+0000","last_undegraded":"2024-03-16T10:06:11.723238+0000","last_fullsized":"2024-03-16T10:06:11.723238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:33:28.909540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415334+0000","last_change":"2024-03-16T10:06:04.221069+0000","last_active":"2024-03-16T10:06:09.415334+0000","last_peered":"2024-03-16T10:06:09.415334+0000","last_clean":"2024-03-16T10:06:09.415334+0000","last_became_active":"2024-03-16T10:06:04.218197+0000","last_became_peered":"2024-03-16T10:06:04.218197+0000","last_unstale":"2024-03-16T10:06:09.415334+0000","last_undegraded":"2024-03-16T10:06:09.415334+0000","last_fullsized":"2024-03-16T10:06:09.415334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:38:25.262182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.552124+0000","last_change":"2024-03-16T10:06:03.221284+0000","last_active":"2024-03-16T10:06:38.552124+0000","last_peered":"2024-03-16T10:06:38.552124+0000","last_clean":"2024-03-16T10:06:38.552124+0000","last_became_active":"2024-03-16T10:06:03.220467+0000","last_became_peered":"2024-03-16T10:06:03.220467+0000","last_unstale":"2024-03-16T10:06:38.552124+0000","last_undegraded":"2024-03-16T10:06:38.552124+0000","last_fullsized":"2024-03-16T10:06:38.552124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:31:19.753996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599230+0000","last_change":"2024-03-16T10:04:45.615656+0000","last_active":"2024-03-16T10:06:09.599230+0000","last_peered":"2024-03-16T10:06:09.599230+0000","last_clean":"2024-03-16T10:06:09.599230+0000","last_became_active":"2024-03-16T10:04:45.615261+0000","last_became_peered":"2024-03-16T10:04:45.615261+0000","last_unstale":"2024-03-16T10:06:09.599230+0000","last_undegraded":"2024-03-16T10:06:09.599230+0000","last_fullsized":"2024-03-16T10:06:09.599230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_clean_scrub_stamp":"2024-03-16T10:00:56.880115+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:06.546840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333566+0000","last_change":"2024-03-16T10:06:08.529863+0000","last_active":"2024-03-16T10:06:09.333566+0000","last_peered":"2024-03-16T10:06:09.333566+0000","last_clean":"2024-03-16T10:06:09.333566+0000","last_became_active":"2024-03-16T10:06:08.528934+0000","last_became_peered":"2024-03-16T10:06:08.528934+0000","last_unstale":"2024-03-16T10:06:09.333566+0000","last_undegraded":"2024-03-16T10:06:09.333566+0000","last_fullsized":"2024-03-16T10:06:09.333566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:40.443983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930375+0000","last_change":"2024-03-16T10:06:04.206776+0000","last_active":"2024-03-16T10:06:09.930375+0000","last_peered":"2024-03-16T10:06:09.930375+0000","last_clean":"2024-03-16T10:06:09.930375+0000","last_became_active":"2024-03-16T10:06:04.206333+0000","last_became_peered":"2024-03-16T10:06:04.206333+0000","last_unstale":"2024-03-16T10:06:09.930375+0000","last_undegraded":"2024-03-16T10:06:09.930375+0000","last_fullsized":"2024-03-16T10:06:09.930375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:00:34.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:37.536579+0000","last_change":"2024-03-16T10:06:03.226921+0000","last_active":"2024-03-16T10:06:37.536579+0000","last_peered":"2024-03-16T10:06:37.536579+0000","last_clean":"2024-03-16T10:06:37.536579+0000","last_became_active":"2024-03-16T10:06:03.226596+0000","last_became_peered":"2024-03-16T10:06:03.226596+0000","last_unstale":"2024-03-16T10:06:37.536579+0000","last_undegraded":"2024-03-16T10:06:37.536579+0000","last_fullsized":"2024-03-16T10:06:37.536579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:03:50.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930497+0000","last_change":"2024-03-16T10:06:08.344432+0000","last_active":"2024-03-16T10:06:09.930497+0000","last_peered":"2024-03-16T10:06:09.930497+0000","last_clean":"2024-03-16T10:06:09.930497+0000","last_became_active":"2024-03-16T10:06:08.343983+0000","last_became_peered":"2024-03-16T10:06:08.343983+0000","last_unstale":"2024-03-16T10:06:09.930497+0000","last_undegraded":"2024-03-16T10:06:09.930497+0000","last_fullsized":"2024-03-16T10:06:09.930497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:41.686700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.573568+0000","last_change":"2024-03-16T10:06:03.232806+0000","last_active":"2024-03-16T10:06:39.573568+0000","last_peered":"2024-03-16T10:06:39.573568+0000","last_clean":"2024-03-16T10:06:39.573568+0000","last_became_active":"2024-03-16T10:06:03.232465+0000","last_became_peered":"2024-03-16T10:06:03.232465+0000","last_unstale":"2024-03-16T10:06:39.573568+0000","last_undegraded":"2024-03-16T10:06:39.573568+0000","last_fullsized":"2024-03-16T10:06:39.573568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:36.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932036+0000","last_change":"2024-03-16T10:06:04.215899+0000","last_active":"2024-03-16T10:06:09.932036+0000","last_peered":"2024-03-16T10:06:09.932036+0000","last_clean":"2024-03-16T10:06:09.932036+0000","last_became_active":"2024-03-16T10:06:04.215474+0000","last_became_peered":"2024-03-16T10:06:04.215474+0000","last_unstale":"2024-03-16T10:06:09.932036+0000","last_undegraded":"2024-03-16T10:06:09.932036+0000","last_fullsized":"2024-03-16T10:06:09.932036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:27:26.663702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415230+0000","last_change":"2024-03-16T10:06:08.530852+0000","last_active":"2024-03-16T10:06:09.415230+0000","last_peered":"2024-03-16T10:06:09.415230+0000","last_clean":"2024-03-16T10:06:09.415230+0000","last_became_active":"2024-03-16T10:06:08.529386+0000","last_became_peered":"2024-03-16T10:06:08.529386+0000","last_unstale":"2024-03-16T10:06:09.415230+0000","last_undegraded":"2024-03-16T10:06:09.415230+0000","last_fullsized":"2024-03-16T10:06:09.415230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:37:17.201323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.238469+0000","last_change":"2024-03-16T10:06:03.244012+0000","last_active":"2024-03-16T10:06:40.238469+0000","last_peered":"2024-03-16T10:06:40.238469+0000","last_clean":"2024-03-16T10:06:40.238469+0000","last_became_active":"2024-03-16T10:06:03.243612+0000","last_became_peered":"2024-03-16T10:06:03.243612+0000","last_unstale":"2024-03-16T10:06:40.238469+0000","last_undegraded":"2024-03-16T10:06:40.238469+0000","last_fullsized":"2024-03-16T10:06:40.238469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:32:34.807111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416162+0000","last_change":"2024-03-16T10:06:04.226678+0000","last_active":"2024-03-16T10:06:09.416162+0000","last_peered":"2024-03-16T10:06:09.416162+0000","last_clean":"2024-03-16T10:06:09.416162+0000","last_became_active":"2024-03-16T10:06:04.226214+0000","last_became_peered":"2024-03-16T10:06:04.226214+0000","last_unstale":"2024-03-16T10:06:09.416162+0000","last_undegraded":"2024-03-16T10:06:09.416162+0000","last_fullsized":"2024-03-16T10:06:09.416162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:05.732310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929744+0000","last_change":"2024-03-16T10:06:08.326985+0000","last_active":"2024-03-16T10:06:09.929744+0000","last_peered":"2024-03-16T10:06:09.929744+0000","last_clean":"2024-03-16T10:06:09.929744+0000","last_became_active":"2024-03-16T10:06:08.326429+0000","last_became_peered":"2024-03-16T10:06:08.326429+0000","last_unstale":"2024-03-16T10:06:09.929744+0000","last_undegraded":"2024-03-16T10:06:09.929744+0000","last_fullsized":"2024-03-16T10:06:09.929744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:44:59.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.515608+0000","last_change":"2024-03-16T10:06:03.232910+0000","last_active":"2024-03-16T10:06:36.515608+0000","last_peered":"2024-03-16T10:06:36.515608+0000","last_clean":"2024-03-16T10:06:36.515608+0000","last_became_active":"2024-03-16T10:06:03.232682+0000","last_became_peered":"2024-03-16T10:06:03.232682+0000","last_unstale":"2024-03-16T10:06:36.515608+0000","last_undegraded":"2024-03-16T10:06:36.515608+0000","last_fullsized":"2024-03-16T10:06:36.515608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:32:56.731620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337356+0000","last_change":"2024-03-16T10:06:04.227189+0000","last_active":"2024-03-16T10:06:09.337356+0000","last_peered":"2024-03-16T10:06:09.337356+0000","last_clean":"2024-03-16T10:06:09.337356+0000","last_became_active":"2024-03-16T10:06:04.226723+0000","last_became_peered":"2024-03-16T10:06:04.226723+0000","last_unstale":"2024-03-16T10:06:09.337356+0000","last_undegraded":"2024-03-16T10:06:09.337356+0000","last_fullsized":"2024-03-16T10:06:09.337356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:47.873938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929470+0000","last_change":"2024-03-16T10:06:08.529316+0000","last_active":"2024-03-16T10:06:09.929470+0000","last_peered":"2024-03-16T10:06:09.929470+0000","last_clean":"2024-03-16T10:06:09.929470+0000","last_became_active":"2024-03-16T10:06:08.528903+0000","last_became_peered":"2024-03-16T10:06:08.528903+0000","last_unstale":"2024-03-16T10:06:09.929470+0000","last_undegraded":"2024-03-16T10:06:09.929470+0000","last_fullsized":"2024-03-16T10:06:09.929470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:19:33.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.519510+0000","last_change":"2024-03-16T10:06:03.233819+0000","last_active":"2024-03-16T10:06:36.519510+0000","last_peered":"2024-03-16T10:06:36.519510+0000","last_clean":"2024-03-16T10:06:36.519510+0000","last_became_active":"2024-03-16T10:06:03.233593+0000","last_became_peered":"2024-03-16T10:06:03.233593+0000","last_unstale":"2024-03-16T10:06:36.519510+0000","last_undegraded":"2024-03-16T10:06:36.519510+0000","last_fullsized":"2024-03-16T10:06:36.519510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:18.847248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930367+0000","last_change":"2024-03-16T10:06:04.192682+0000","last_active":"2024-03-16T10:06:09.930367+0000","last_peered":"2024-03-16T10:06:09.930367+0000","last_clean":"2024-03-16T10:06:09.930367+0000","last_became_active":"2024-03-16T10:06:04.192305+0000","last_became_peered":"2024-03-16T10:06:04.192305+0000","last_unstale":"2024-03-16T10:06:09.930367+0000","last_undegraded":"2024-03-16T10:06:09.930367+0000","last_fullsized":"2024-03-16T10:06:09.930367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:46.718663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931685+0000","last_change":"2024-03-16T10:06:08.355342+0000","last_active":"2024-03-16T10:06:09.931685+0000","last_peered":"2024-03-16T10:06:09.931685+0000","last_clean":"2024-03-16T10:06:09.931685+0000","last_became_active":"2024-03-16T10:06:08.351547+0000","last_became_peered":"2024-03-16T10:06:08.351547+0000","last_unstale":"2024-03-16T10:06:09.931685+0000","last_undegraded":"2024-03-16T10:06:09.931685+0000","last_fullsized":"2024-03-16T10:06:09.931685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:01:00.777949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.576986+0000","last_change":"2024-03-16T10:06:03.242248+0000","last_active":"2024-03-16T10:06:39.576986+0000","last_peered":"2024-03-16T10:06:39.576986+0000","last_clean":"2024-03-16T10:06:39.576986+0000","last_became_active":"2024-03-16T10:06:03.241842+0000","last_became_peered":"2024-03-16T10:06:03.241842+0000","last_unstale":"2024-03-16T10:06:39.576986+0000","last_undegraded":"2024-03-16T10:06:39.576986+0000","last_fullsized":"2024-03-16T10:06:39.576986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:03:35.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930103+0000","last_change":"2024-03-16T10:06:04.221790+0000","last_active":"2024-03-16T10:06:09.930103+0000","last_peered":"2024-03-16T10:06:09.930103+0000","last_clean":"2024-03-16T10:06:09.930103+0000","last_became_active":"2024-03-16T10:06:04.221475+0000","last_became_peered":"2024-03-16T10:06:04.221475+0000","last_unstale":"2024-03-16T10:06:09.930103+0000","last_undegraded":"2024-03-16T10:06:09.930103+0000","last_fullsized":"2024-03-16T10:06:09.930103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:05:50.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929373+0000","last_change":"2024-03-16T10:06:08.337906+0000","last_active":"2024-03-16T10:06:09.929373+0000","last_peered":"2024-03-16T10:06:09.929373+0000","last_clean":"2024-03-16T10:06:09.929373+0000","last_became_active":"2024-03-16T10:06:08.337444+0000","last_became_peered":"2024-03-16T10:06:08.337444+0000","last_unstale":"2024-03-16T10:06:09.929373+0000","last_undegraded":"2024-03-16T10:06:09.929373+0000","last_fullsized":"2024-03-16T10:06:09.929373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:09:03.997606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.557289+0000","last_change":"2024-03-16T10:06:03.235717+0000","last_active":"2024-03-16T10:06:38.557289+0000","last_peered":"2024-03-16T10:06:38.557289+0000","last_clean":"2024-03-16T10:06:38.557289+0000","last_became_active":"2024-03-16T10:06:03.235331+0000","last_became_peered":"2024-03-16T10:06:03.235331+0000","last_unstale":"2024-03-16T10:06:38.557289+0000","last_undegraded":"2024-03-16T10:06:38.557289+0000","last_fullsized":"2024-03-16T10:06:38.557289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:08.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415734+0000","last_change":"2024-03-16T10:06:04.229219+0000","last_active":"2024-03-16T10:06:09.415734+0000","last_peered":"2024-03-16T10:06:09.415734+0000","last_clean":"2024-03-16T10:06:09.415734+0000","last_became_active":"2024-03-16T10:06:04.226850+0000","last_became_peered":"2024-03-16T10:06:04.226850+0000","last_unstale":"2024-03-16T10:06:09.415734+0000","last_undegraded":"2024-03-16T10:06:09.415734+0000","last_fullsized":"2024-03-16T10:06:09.415734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:22:48.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929510+0000","last_change":"2024-03-16T10:06:08.529201+0000","last_active":"2024-03-16T10:06:09.929510+0000","last_peered":"2024-03-16T10:06:09.929510+0000","last_clean":"2024-03-16T10:06:09.929510+0000","last_became_active":"2024-03-16T10:06:08.528782+0000","last_became_peered":"2024-03-16T10:06:08.528782+0000","last_unstale":"2024-03-16T10:06:09.929510+0000","last_undegraded":"2024-03-16T10:06:09.929510+0000","last_fullsized":"2024-03-16T10:06:09.929510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:35:38.267168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.515699+0000","last_change":"2024-03-16T10:06:03.227393+0000","last_active":"2024-03-16T10:06:36.515699+0000","last_peered":"2024-03-16T10:06:36.515699+0000","last_clean":"2024-03-16T10:06:36.515699+0000","last_became_active":"2024-03-16T10:06:03.227085+0000","last_became_peered":"2024-03-16T10:06:03.227085+0000","last_unstale":"2024-03-16T10:06:36.515699+0000","last_undegraded":"2024-03-16T10:06:36.515699+0000","last_fullsized":"2024-03-16T10:06:36.515699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:04.302592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599457+0000","last_change":"2024-03-16T10:06:04.217699+0000","last_active":"2024-03-16T10:06:09.599457+0000","last_peered":"2024-03-16T10:06:09.599457+0000","last_clean":"2024-03-16T10:06:09.599457+0000","last_became_active":"2024-03-16T10:06:04.217378+0000","last_became_peered":"2024-03-16T10:06:04.217378+0000","last_unstale":"2024-03-16T10:06:09.599457+0000","last_undegraded":"2024-03-16T10:06:09.599457+0000","last_fullsized":"2024-03-16T10:06:09.599457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:35:23.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333378+0000","last_change":"2024-03-16T10:06:08.344488+0000","last_active":"2024-03-16T10:06:09.333378+0000","last_peered":"2024-03-16T10:06:09.333378+0000","last_clean":"2024-03-16T10:06:09.333378+0000","last_became_active":"2024-03-16T10:06:08.343964+0000","last_became_peered":"2024-03-16T10:06:08.343964+0000","last_unstale":"2024-03-16T10:06:09.333378+0000","last_undegraded":"2024-03-16T10:06:09.333378+0000","last_fullsized":"2024-03-16T10:06:09.333378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:41:09.144062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.602741+0000","last_change":"2024-03-16T10:06:03.231805+0000","last_active":"2024-03-16T10:06:40.602741+0000","last_peered":"2024-03-16T10:06:40.602741+0000","last_clean":"2024-03-16T10:06:40.602741+0000","last_became_active":"2024-03-16T10:06:03.230605+0000","last_became_peered":"2024-03-16T10:06:03.230605+0000","last_unstale":"2024-03-16T10:06:40.602741+0000","last_undegraded":"2024-03-16T10:06:40.602741+0000","last_fullsized":"2024-03-16T10:06:40.602741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:21.545277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599913+0000","last_change":"2024-03-16T10:06:04.195674+0000","last_active":"2024-03-16T10:06:09.599913+0000","last_peered":"2024-03-16T10:06:09.599913+0000","last_clean":"2024-03-16T10:06:09.599913+0000","last_became_active":"2024-03-16T10:06:04.195364+0000","last_became_peered":"2024-03-16T10:06:04.195364+0000","last_unstale":"2024-03-16T10:06:09.599913+0000","last_undegraded":"2024-03-16T10:06:09.599913+0000","last_fullsized":"2024-03-16T10:06:09.599913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:40.101280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336455+0000","last_change":"2024-03-16T10:06:08.342352+0000","last_active":"2024-03-16T10:06:09.336455+0000","last_peered":"2024-03-16T10:06:09.336455+0000","last_clean":"2024-03-16T10:06:09.336455+0000","last_became_active":"2024-03-16T10:06:08.341910+0000","last_became_peered":"2024-03-16T10:06:08.341910+0000","last_unstale":"2024-03-16T10:06:09.336455+0000","last_undegraded":"2024-03-16T10:06:09.336455+0000","last_fullsized":"2024-03-16T10:06:09.336455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:23.446076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333887+0000","last_change":"2024-03-16T10:06:03.236814+0000","last_active":"2024-03-16T10:06:09.333887+0000","last_peered":"2024-03-16T10:06:09.333887+0000","last_clean":"2024-03-16T10:06:09.333887+0000","last_became_active":"2024-03-16T10:06:03.236428+0000","last_became_peered":"2024-03-16T10:06:03.236428+0000","last_unstale":"2024-03-16T10:06:09.333887+0000","last_undegraded":"2024-03-16T10:06:09.333887+0000","last_fullsized":"2024-03-16T10:06:09.333887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:44.565924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599690+0000","last_change":"2024-03-16T10:06:04.224725+0000","last_active":"2024-03-16T10:06:09.599690+0000","last_peered":"2024-03-16T10:06:09.599690+0000","last_clean":"2024-03-16T10:06:09.599690+0000","last_became_active":"2024-03-16T10:06:04.224308+0000","last_became_peered":"2024-03-16T10:06:04.224308+0000","last_unstale":"2024-03-16T10:06:09.599690+0000","last_undegraded":"2024-03-16T10:06:09.599690+0000","last_fullsized":"2024-03-16T10:06:09.599690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:55.128000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.332993+0000","last_change":"2024-03-16T10:06:08.330089+0000","last_active":"2024-03-16T10:06:09.332993+0000","last_peered":"2024-03-16T10:06:09.332993+0000","last_clean":"2024-03-16T10:06:09.332993+0000","last_became_active":"2024-03-16T10:06:08.329692+0000","last_became_peered":"2024-03-16T10:06:08.329692+0000","last_unstale":"2024-03-16T10:06:09.332993+0000","last_undegraded":"2024-03-16T10:06:09.332993+0000","last_fullsized":"2024-03-16T10:06:09.332993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:22.141351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.577706+0000","last_change":"2024-03-16T10:06:03.247525+0000","last_active":"2024-03-16T10:06:39.577706+0000","last_peered":"2024-03-16T10:06:39.577706+0000","last_clean":"2024-03-16T10:06:39.577706+0000","last_became_active":"2024-03-16T10:06:03.246854+0000","last_became_peered":"2024-03-16T10:06:03.246854+0000","last_unstale":"2024-03-16T10:06:39.577706+0000","last_undegraded":"2024-03-16T10:06:39.577706+0000","last_fullsized":"2024-03-16T10:06:39.577706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:20.648750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599883+0000","last_change":"2024-03-16T10:06:04.194982+0000","last_active":"2024-03-16T10:06:09.599883+0000","last_peered":"2024-03-16T10:06:09.599883+0000","last_clean":"2024-03-16T10:06:09.599883+0000","last_became_active":"2024-03-16T10:06:04.194696+0000","last_became_peered":"2024-03-16T10:06:04.194696+0000","last_unstale":"2024-03-16T10:06:09.599883+0000","last_undegraded":"2024-03-16T10:06:09.599883+0000","last_fullsized":"2024-03-16T10:06:09.599883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:20.085100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930237+0000","last_change":"2024-03-16T10:06:04.196047+0000","last_active":"2024-03-16T10:06:09.930237+0000","last_peered":"2024-03-16T10:06:09.930237+0000","last_clean":"2024-03-16T10:06:09.930237+0000","last_became_active":"2024-03-16T10:06:04.195742+0000","last_became_peered":"2024-03-16T10:06:04.195742+0000","last_unstale":"2024-03-16T10:06:09.930237+0000","last_undegraded":"2024-03-16T10:06:09.930237+0000","last_fullsized":"2024-03-16T10:06:09.930237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:00:54.239827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930677+0000","last_change":"2024-03-16T10:06:04.193481+0000","last_active":"2024-03-16T10:06:09.930677+0000","last_peered":"2024-03-16T10:06:09.930677+0000","last_clean":"2024-03-16T10:06:09.930677+0000","last_became_active":"2024-03-16T10:06:04.193145+0000","last_became_peered":"2024-03-16T10:06:04.193145+0000","last_unstale":"2024-03-16T10:06:09.930677+0000","last_undegraded":"2024-03-16T10:06:09.930677+0000","last_fullsized":"2024-03-16T10:06:09.930677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:02:10.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929783+0000","last_change":"2024-03-16T10:06:04.223322+0000","last_active":"2024-03-16T10:06:09.929783+0000","last_peered":"2024-03-16T10:06:09.929783+0000","last_clean":"2024-03-16T10:06:09.929783+0000","last_became_active":"2024-03-16T10:06:04.223116+0000","last_became_peered":"2024-03-16T10:06:04.223116+0000","last_unstale":"2024-03-16T10:06:09.929783+0000","last_undegraded":"2024-03-16T10:06:09.929783+0000","last_fullsized":"2024-03-16T10:06:09.929783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:46.641716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336831+0000","last_change":"2024-03-16T10:06:04.221926+0000","last_active":"2024-03-16T10:06:09.336831+0000","last_peered":"2024-03-16T10:06:09.336831+0000","last_clean":"2024-03-16T10:06:09.336831+0000","last_became_active":"2024-03-16T10:06:04.221427+0000","last_became_peered":"2024-03-16T10:06:04.221427+0000","last_unstale":"2024-03-16T10:06:09.336831+0000","last_undegraded":"2024-03-16T10:06:09.336831+0000","last_fullsized":"2024-03-16T10:06:09.336831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:51.230757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932117+0000","last_change":"2024-03-16T10:06:04.225518+0000","last_active":"2024-03-16T10:06:09.932117+0000","last_peered":"2024-03-16T10:06:09.932117+0000","last_clean":"2024-03-16T10:06:09.932117+0000","last_became_active":"2024-03-16T10:06:04.225102+0000","last_became_peered":"2024-03-16T10:06:04.225102+0000","last_unstale":"2024-03-16T10:06:09.932117+0000","last_undegraded":"2024-03-16T10:06:09.932117+0000","last_fullsized":"2024-03-16T10:06:09.932117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:38:54.493238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930756+0000","last_change":"2024-03-16T10:06:04.192978+0000","last_active":"2024-03-16T10:06:09.930756+0000","last_peered":"2024-03-16T10:06:09.930756+0000","last_clean":"2024-03-16T10:06:09.930756+0000","last_became_active":"2024-03-16T10:06:04.192641+0000","last_became_peered":"2024-03-16T10:06:04.192641+0000","last_unstale":"2024-03-16T10:06:09.930756+0000","last_undegraded":"2024-03-16T10:06:09.930756+0000","last_fullsized":"2024-03-16T10:06:09.930756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:58.334266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931573+0000","last_change":"2024-03-16T10:06:04.190020+0000","last_active":"2024-03-16T10:06:09.931573+0000","last_peered":"2024-03-16T10:06:09.931573+0000","last_clean":"2024-03-16T10:06:09.931573+0000","last_became_active":"2024-03-16T10:06:04.189703+0000","last_became_peered":"2024-03-16T10:06:04.189703+0000","last_unstale":"2024-03-16T10:06:09.931573+0000","last_undegraded":"2024-03-16T10:06:09.931573+0000","last_fullsized":"2024-03-16T10:06:09.931573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:23.689558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416311+0000","last_change":"2024-03-16T10:06:04.234714+0000","last_active":"2024-03-16T10:06:09.416311+0000","last_peered":"2024-03-16T10:06:09.416311+0000","last_clean":"2024-03-16T10:06:09.416311+0000","last_became_active":"2024-03-16T10:06:04.233246+0000","last_became_peered":"2024-03-16T10:06:04.233246+0000","last_unstale":"2024-03-16T10:06:09.416311+0000","last_undegraded":"2024-03-16T10:06:09.416311+0000","last_fullsized":"2024-03-16T10:06:09.416311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:13:31.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333336+0000","last_change":"2024-03-16T10:06:04.219242+0000","last_active":"2024-03-16T10:06:09.333336+0000","last_peered":"2024-03-16T10:06:09.333336+0000","last_clean":"2024-03-16T10:06:09.333336+0000","last_became_active":"2024-03-16T10:06:04.218955+0000","last_became_peered":"2024-03-16T10:06:04.218955+0000","last_unstale":"2024-03-16T10:06:09.333336+0000","last_undegraded":"2024-03-16T10:06:09.333336+0000","last_fullsized":"2024-03-16T10:06:09.333336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:40.760143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415450+0000","last_change":"2024-03-16T10:06:04.231516+0000","last_active":"2024-03-16T10:06:09.415450+0000","last_peered":"2024-03-16T10:06:09.415450+0000","last_clean":"2024-03-16T10:06:09.415450+0000","last_became_active":"2024-03-16T10:06:04.230328+0000","last_became_peered":"2024-03-16T10:06:04.230328+0000","last_unstale":"2024-03-16T10:06:09.415450+0000","last_undegraded":"2024-03-16T10:06:09.415450+0000","last_fullsized":"2024-03-16T10:06:09.415450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:48:55.678921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333288+0000","last_change":"2024-03-16T10:06:04.219222+0000","last_active":"2024-03-16T10:06:09.333288+0000","last_peered":"2024-03-16T10:06:09.333288+0000","last_clean":"2024-03-16T10:06:09.333288+0000","last_became_active":"2024-03-16T10:06:04.218903+0000","last_became_peered":"2024-03-16T10:06:04.218903+0000","last_unstale":"2024-03-16T10:06:09.333288+0000","last_undegraded":"2024-03-16T10:06:09.333288+0000","last_fullsized":"2024-03-16T10:06:09.333288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:23.210739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932090+0000","last_change":"2024-03-16T10:06:04.226025+0000","last_active":"2024-03-16T10:06:09.932090+0000","last_peered":"2024-03-16T10:06:09.932090+0000","last_clean":"2024-03-16T10:06:09.932090+0000","last_became_active":"2024-03-16T10:06:04.225657+0000","last_became_peered":"2024-03-16T10:06:04.225657+0000","last_unstale":"2024-03-16T10:06:09.932090+0000","last_undegraded":"2024-03-16T10:06:09.932090+0000","last_fullsized":"2024-03-16T10:06:09.932090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:08:15.163902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930517+0000","last_change":"2024-03-16T10:06:04.198047+0000","last_active":"2024-03-16T10:06:09.930517+0000","last_peered":"2024-03-16T10:06:09.930517+0000","last_clean":"2024-03-16T10:06:09.930517+0000","last_became_active":"2024-03-16T10:06:04.197751+0000","last_became_peered":"2024-03-16T10:06:04.197751+0000","last_unstale":"2024-03-16T10:06:09.930517+0000","last_undegraded":"2024-03-16T10:06:09.930517+0000","last_fullsized":"2024-03-16T10:06:09.930517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:52.703845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":215634,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48013,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":217,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":85498,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1095902,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":485484,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":485862,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1082359,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":480988,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480960,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":505770,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480696,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480932,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":44033,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:06:43.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:06:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 -- 172.21.15.39:0/1403690657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac101620 msgr2=0x7f44ac101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 --2- 172.21.15.39:0/1403690657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac101620 0x7f44ac101a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f449c009860 tx=0x7f449c02bb70 comp rx=0 tx=0).stop 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 -- 172.21.15.39:0/1403690657 shutdown_connections 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 --2- 172.21.15.39:0/1403690657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac1031d0 0x7f44ac109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 --2- 172.21.15.39:0/1403690657 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac102810 0x7f44ac102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 --2- 172.21.15.39:0/1403690657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac101620 0x7f44ac101a40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 -- 172.21.15.39:0/1403690657 >> 172.21.15.39:0/1403690657 conn(0x7f44ac0fcb90 msgr2=0x7f44ac0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 -- 172.21.15.39:0/1403690657 shutdown_connections 2024-03-16T10:06:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.485+0000 7f44b1c00640 1 -- 172.21.15.39:0/1403690657 wait complete. 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 Processor -- start 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 -- start start 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac102810 0x7f44ac073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac10d290 con 0x7f44ac102810 2024-03-16T10:06:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac10d110 con 0x7f44ac1031d0 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.486+0000 7f44b1c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac10d410 con 0x7f44ac101620 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44b1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44b1200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40856/0 (socket says 172.21.15.39:40856) 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44b1200640 1 -- 172.21.15.39:0/3936303881 learned_addr learned my addr 172.21.15.39:0/3936303881 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47736/0 (socket says 172.21.15.39:47736) 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44ab400640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac102810 0x7f44ac073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 -- 172.21.15.39:0/3936303881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 msgr2=0x7f44ac074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 -- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac102810 msgr2=0x7f44ac073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac102810 0x7f44ac073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 -- 172.21.15.39:0/3936303881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac0749a0 con 0x7f44ac101620 2024-03-16T10:06:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44b1200640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.487+0000 7f44abe00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f449c009780 tx=0x7f449c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.488+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c041020 con 0x7f44ac101620 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.488+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c0323b0 con 0x7f44ac101620 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.488+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac1a1990 con 0x7f44ac101620 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.488+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c039040 con 0x7f44ac101620 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.488+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1a1eb0 con 0x7f44ac101620 2024-03-16T10:06:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.490+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac108940 con 0x7f44ac101620 2024-03-16T10:06:43.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.491+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f449c044020 con 0x7f44ac101620 2024-03-16T10:06:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.492+0000 7f44a8c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 0x7f44900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.492+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f449c0f0690 con 0x7f44ac101620 2024-03-16T10:06:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.492+0000 7f44ab400640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 0x7f44900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.493+0000 7f44ab400640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 0x7f44900a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44ac075300 tx=0x7f4498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:43.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.496+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f449c04b020 con 0x7f44ac101620 2024-03-16T10:06:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.700+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44ac061c20 con 0x7f44900a2890 2024-03-16T10:06:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.712+0000 7f44a8c00640 1 -- 172.21.15.39:0/3936303881 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340650 (secure 0 0 0) 0x7f44ac061c20 con 0x7f44900a2890 2024-03-16T10:06:43.718 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.717+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 msgr2=0x7f44900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.717+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 0x7f44900a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44ac075300 tx=0x7f4498009260 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.717+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 msgr2=0x7f44ac075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.717+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f449c009780 tx=0x7f449c0042f0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 shutdown_connections 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44900a2890 0x7f44900a4d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44ac1031d0 0x7f44ac074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44ac102810 0x7f44ac073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 --2- 172.21.15.39:0/3936303881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44ac101620 0x7f44ac075590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 >> 172.21.15.39:0/3936303881 conn(0x7f44ac0fcb90 msgr2=0x7f44ac105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 shutdown_connections 2024-03-16T10:06:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:43.718+0000 7f44b1c00640 1 -- 172.21.15.39:0/3936303881 wait complete. 2024-03-16T10:06:43.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-03-16T10:06:42.213582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638498,"num_objects":95,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":168,"num_read_kb":185,"num_write":399,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352712,"kb_used_data":11876,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589112,"statfs":{"total":1151940427776,"available":1151579250688,"internally_reserved":0,"allocated":12161024,"data_stored":7614725,"data_compressed":285735,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43631,"num_objects":71,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":170,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931188+0000","last_change":"2024-03-16T10:06:04.221459+0000","last_active":"2024-03-16T10:06:09.931188+0000","last_peered":"2024-03-16T10:06:09.931188+0000","last_clean":"2024-03-16T10:06:09.931188+0000","last_became_active":"2024-03-16T10:06:04.221172+0000","last_became_peered":"2024-03-16T10:06:04.221172+0000","last_unstale":"2024-03-16T10:06:09.931188+0000","last_undegraded":"2024-03-16T10:06:09.931188+0000","last_fullsized":"2024-03-16T10:06:09.931188+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:58:28.019985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337431+0000","last_change":"2024-03-16T10:06:04.205876+0000","last_active":"2024-03-16T10:06:09.337431+0000","last_peered":"2024-03-16T10:06:09.337431+0000","last_clean":"2024-03-16T10:06:09.337431+0000","last_became_active":"2024-03-16T10:06:04.205473+0000","last_became_peered":"2024-03-16T10:06:04.205473+0000","last_unstale":"2024-03-16T10:06:09.337431+0000","last_undegraded":"2024-03-16T10:06:09.337431+0000","last_fullsized":"2024-03-16T10:06:09.337431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:37.981148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336941+0000","last_change":"2024-03-16T10:06:04.197872+0000","last_active":"2024-03-16T10:06:09.336941+0000","last_peered":"2024-03-16T10:06:09.336941+0000","last_clean":"2024-03-16T10:06:09.336941+0000","last_became_active":"2024-03-16T10:06:04.197347+0000","last_became_peered":"2024-03-16T10:06:04.197347+0000","last_unstale":"2024-03-16T10:06:09.336941+0000","last_undegraded":"2024-03-16T10:06:09.336941+0000","last_fullsized":"2024-03-16T10:06:09.336941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:37.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929702+0000","last_change":"2024-03-16T10:06:04.196359+0000","last_active":"2024-03-16T10:06:09.929702+0000","last_peered":"2024-03-16T10:06:09.929702+0000","last_clean":"2024-03-16T10:06:09.929702+0000","last_became_active":"2024-03-16T10:06:04.196072+0000","last_became_peered":"2024-03-16T10:06:04.196072+0000","last_unstale":"2024-03-16T10:06:09.929702+0000","last_undegraded":"2024-03-16T10:06:09.929702+0000","last_fullsized":"2024-03-16T10:06:09.929702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:03.641536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930496+0000","last_change":"2024-03-16T10:06:04.224386+0000","last_active":"2024-03-16T10:06:09.930496+0000","last_peered":"2024-03-16T10:06:09.930496+0000","last_clean":"2024-03-16T10:06:09.930496+0000","last_became_active":"2024-03-16T10:06:04.224042+0000","last_became_peered":"2024-03-16T10:06:04.224042+0000","last_unstale":"2024-03-16T10:06:09.930496+0000","last_undegraded":"2024-03-16T10:06:09.930496+0000","last_fullsized":"2024-03-16T10:06:09.930496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:16:50.114249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930098+0000","last_change":"2024-03-16T10:06:04.214055+0000","last_active":"2024-03-16T10:06:09.930098+0000","last_peered":"2024-03-16T10:06:09.930098+0000","last_clean":"2024-03-16T10:06:09.930098+0000","last_became_active":"2024-03-16T10:06:04.213792+0000","last_became_peered":"2024-03-16T10:06:04.213792+0000","last_unstale":"2024-03-16T10:06:09.930098+0000","last_undegraded":"2024-03-16T10:06:09.930098+0000","last_fullsized":"2024-03-16T10:06:09.930098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:50.142741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599348+0000","last_change":"2024-03-16T10:06:04.208874+0000","last_active":"2024-03-16T10:06:09.599348+0000","last_peered":"2024-03-16T10:06:09.599348+0000","last_clean":"2024-03-16T10:06:09.599348+0000","last_became_active":"2024-03-16T10:06:04.208553+0000","last_became_peered":"2024-03-16T10:06:04.208553+0000","last_unstale":"2024-03-16T10:06:09.599348+0000","last_undegraded":"2024-03-16T10:06:09.599348+0000","last_fullsized":"2024-03-16T10:06:09.599348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:33.885089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415216+0000","last_change":"2024-03-16T10:06:04.233487+0000","last_active":"2024-03-16T10:06:09.415216+0000","last_peered":"2024-03-16T10:06:09.415216+0000","last_clean":"2024-03-16T10:06:09.415216+0000","last_became_active":"2024-03-16T10:06:04.228530+0000","last_became_peered":"2024-03-16T10:06:04.228530+0000","last_unstale":"2024-03-16T10:06:09.415216+0000","last_undegraded":"2024-03-16T10:06:09.415216+0000","last_fullsized":"2024-03-16T10:06:09.415216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:53:12.270229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333665+0000","last_change":"2024-03-16T10:06:04.190965+0000","last_active":"2024-03-16T10:06:09.333665+0000","last_peered":"2024-03-16T10:06:09.333665+0000","last_clean":"2024-03-16T10:06:09.333665+0000","last_became_active":"2024-03-16T10:06:04.190681+0000","last_became_peered":"2024-03-16T10:06:04.190681+0000","last_unstale":"2024-03-16T10:06:09.333665+0000","last_undegraded":"2024-03-16T10:06:09.333665+0000","last_fullsized":"2024-03-16T10:06:09.333665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:46.498680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930187+0000","last_change":"2024-03-16T10:06:04.198098+0000","last_active":"2024-03-16T10:06:09.930187+0000","last_peered":"2024-03-16T10:06:09.930187+0000","last_clean":"2024-03-16T10:06:09.930187+0000","last_became_active":"2024-03-16T10:06:04.197758+0000","last_became_peered":"2024-03-16T10:06:04.197758+0000","last_unstale":"2024-03-16T10:06:09.930187+0000","last_undegraded":"2024-03-16T10:06:09.930187+0000","last_fullsized":"2024-03-16T10:06:09.930187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:27.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599821+0000","last_change":"2024-03-16T10:06:04.195970+0000","last_active":"2024-03-16T10:06:09.599821+0000","last_peered":"2024-03-16T10:06:09.599821+0000","last_clean":"2024-03-16T10:06:09.599821+0000","last_became_active":"2024-03-16T10:06:04.195522+0000","last_became_peered":"2024-03-16T10:06:04.195522+0000","last_unstale":"2024-03-16T10:06:09.599821+0000","last_undegraded":"2024-03-16T10:06:09.599821+0000","last_fullsized":"2024-03-16T10:06:09.599821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:22.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929573+0000","last_change":"2024-03-16T10:06:04.196075+0000","last_active":"2024-03-16T10:06:09.929573+0000","last_peered":"2024-03-16T10:06:09.929573+0000","last_clean":"2024-03-16T10:06:09.929573+0000","last_became_active":"2024-03-16T10:06:04.195785+0000","last_became_peered":"2024-03-16T10:06:04.195785+0000","last_unstale":"2024-03-16T10:06:09.929573+0000","last_undegraded":"2024-03-16T10:06:09.929573+0000","last_fullsized":"2024-03-16T10:06:09.929573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:52:50.016787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932113+0000","last_change":"2024-03-16T10:06:04.206173+0000","last_active":"2024-03-16T10:06:09.932113+0000","last_peered":"2024-03-16T10:06:09.932113+0000","last_clean":"2024-03-16T10:06:09.932113+0000","last_became_active":"2024-03-16T10:06:04.205766+0000","last_became_peered":"2024-03-16T10:06:04.205766+0000","last_unstale":"2024-03-16T10:06:09.932113+0000","last_undegraded":"2024-03-16T10:06:09.932113+0000","last_fullsized":"2024-03-16T10:06:09.932113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:25:47.405044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336131+0000","last_change":"2024-03-16T10:06:04.196204+0000","last_active":"2024-03-16T10:06:09.336131+0000","last_peered":"2024-03-16T10:06:09.336131+0000","last_clean":"2024-03-16T10:06:09.336131+0000","last_became_active":"2024-03-16T10:06:04.194255+0000","last_became_peered":"2024-03-16T10:06:04.194255+0000","last_unstale":"2024-03-16T10:06:09.336131+0000","last_undegraded":"2024-03-16T10:06:09.336131+0000","last_fullsized":"2024-03-16T10:06:09.336131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:44.367876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.335954+0000","last_change":"2024-03-16T10:06:04.203298+0000","last_active":"2024-03-16T10:06:09.335954+0000","last_peered":"2024-03-16T10:06:09.335954+0000","last_clean":"2024-03-16T10:06:09.335954+0000","last_became_active":"2024-03-16T10:06:04.202790+0000","last_became_peered":"2024-03-16T10:06:04.202790+0000","last_unstale":"2024-03-16T10:06:09.335954+0000","last_undegraded":"2024-03-16T10:06:09.335954+0000","last_fullsized":"2024-03-16T10:06:09.335954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:27.360861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.332951+0000","last_change":"2024-03-16T10:06:04.191196+0000","last_active":"2024-03-16T10:06:09.332951+0000","last_peered":"2024-03-16T10:06:09.332951+0000","last_clean":"2024-03-16T10:06:09.332951+0000","last_became_active":"2024-03-16T10:06:04.190873+0000","last_became_peered":"2024-03-16T10:06:04.190873+0000","last_unstale":"2024-03-16T10:06:09.332951+0000","last_undegraded":"2024-03-16T10:06:09.332951+0000","last_fullsized":"2024-03-16T10:06:09.332951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:59.272388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930962+0000","last_change":"2024-03-16T10:06:04.221443+0000","last_active":"2024-03-16T10:06:09.930962+0000","last_peered":"2024-03-16T10:06:09.930962+0000","last_clean":"2024-03-16T10:06:09.930962+0000","last_became_active":"2024-03-16T10:06:04.221182+0000","last_became_peered":"2024-03-16T10:06:04.221182+0000","last_unstale":"2024-03-16T10:06:09.930962+0000","last_undegraded":"2024-03-16T10:06:09.930962+0000","last_fullsized":"2024-03-16T10:06:09.930962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:04:08.139784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931790+0000","last_change":"2024-03-16T10:06:04.208876+0000","last_active":"2024-03-16T10:06:09.931790+0000","last_peered":"2024-03-16T10:06:09.931790+0000","last_clean":"2024-03-16T10:06:09.931790+0000","last_became_active":"2024-03-16T10:06:04.208361+0000","last_became_peered":"2024-03-16T10:06:04.208361+0000","last_unstale":"2024-03-16T10:06:09.931790+0000","last_undegraded":"2024-03-16T10:06:09.931790+0000","last_fullsized":"2024-03-16T10:06:09.931790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:53.156832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931108+0000","last_change":"2024-03-16T10:06:04.228080+0000","last_active":"2024-03-16T10:06:09.931108+0000","last_peered":"2024-03-16T10:06:09.931108+0000","last_clean":"2024-03-16T10:06:09.931108+0000","last_became_active":"2024-03-16T10:06:04.227799+0000","last_became_peered":"2024-03-16T10:06:04.227799+0000","last_unstale":"2024-03-16T10:06:09.931108+0000","last_undegraded":"2024-03-16T10:06:09.931108+0000","last_fullsized":"2024-03-16T10:06:09.931108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:40:55.113354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930478+0000","last_change":"2024-03-16T10:06:08.349744+0000","last_active":"2024-03-16T10:06:09.930478+0000","last_peered":"2024-03-16T10:06:09.930478+0000","last_clean":"2024-03-16T10:06:09.930478+0000","last_became_active":"2024-03-16T10:06:08.349359+0000","last_became_peered":"2024-03-16T10:06:08.349359+0000","last_unstale":"2024-03-16T10:06:09.930478+0000","last_undegraded":"2024-03-16T10:06:09.930478+0000","last_fullsized":"2024-03-16T10:06:09.930478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:30:02.614189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.600329+0000","last_change":"2024-03-16T10:06:03.234836+0000","last_active":"2024-03-16T10:06:40.600329+0000","last_peered":"2024-03-16T10:06:40.600329+0000","last_clean":"2024-03-16T10:06:40.600329+0000","last_became_active":"2024-03-16T10:06:03.234436+0000","last_became_peered":"2024-03-16T10:06:03.234436+0000","last_unstale":"2024-03-16T10:06:40.600329+0000","last_undegraded":"2024-03-16T10:06:40.600329+0000","last_fullsized":"2024-03-16T10:06:40.600329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:52:34.069875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416167+0000","last_change":"2024-03-16T10:06:04.222012+0000","last_active":"2024-03-16T10:06:09.416167+0000","last_peered":"2024-03-16T10:06:09.416167+0000","last_clean":"2024-03-16T10:06:09.416167+0000","last_became_active":"2024-03-16T10:06:04.221661+0000","last_became_peered":"2024-03-16T10:06:04.221661+0000","last_unstale":"2024-03-16T10:06:09.416167+0000","last_undegraded":"2024-03-16T10:06:09.416167+0000","last_fullsized":"2024-03-16T10:06:09.416167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:07.837752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929109+0000","last_change":"2024-03-16T10:06:08.345658+0000","last_active":"2024-03-16T10:06:09.929109+0000","last_peered":"2024-03-16T10:06:09.929109+0000","last_clean":"2024-03-16T10:06:09.929109+0000","last_became_active":"2024-03-16T10:06:08.345291+0000","last_became_peered":"2024-03-16T10:06:08.345291+0000","last_unstale":"2024-03-16T10:06:09.929109+0000","last_undegraded":"2024-03-16T10:06:09.929109+0000","last_fullsized":"2024-03-16T10:06:09.929109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:43:39.734164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416279+0000","last_change":"2024-03-16T10:06:03.244368+0000","last_active":"2024-03-16T10:06:09.416279+0000","last_peered":"2024-03-16T10:06:09.416279+0000","last_clean":"2024-03-16T10:06:09.416279+0000","last_became_active":"2024-03-16T10:06:03.243369+0000","last_became_peered":"2024-03-16T10:06:03.243369+0000","last_unstale":"2024-03-16T10:06:09.416279+0000","last_undegraded":"2024-03-16T10:06:09.416279+0000","last_fullsized":"2024-03-16T10:06:09.416279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:38.473988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337432+0000","last_change":"2024-03-16T10:06:04.217255+0000","last_active":"2024-03-16T10:06:09.337432+0000","last_peered":"2024-03-16T10:06:09.337432+0000","last_clean":"2024-03-16T10:06:09.337432+0000","last_became_active":"2024-03-16T10:06:04.216659+0000","last_became_peered":"2024-03-16T10:06:04.216659+0000","last_unstale":"2024-03-16T10:06:09.337432+0000","last_undegraded":"2024-03-16T10:06:09.337432+0000","last_fullsized":"2024-03-16T10:06:09.337432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:06.081507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929293+0000","last_change":"2024-03-16T10:06:08.346622+0000","last_active":"2024-03-16T10:06:09.929293+0000","last_peered":"2024-03-16T10:06:09.929293+0000","last_clean":"2024-03-16T10:06:09.929293+0000","last_became_active":"2024-03-16T10:06:08.346209+0000","last_became_peered":"2024-03-16T10:06:08.346209+0000","last_unstale":"2024-03-16T10:06:09.929293+0000","last_undegraded":"2024-03-16T10:06:09.929293+0000","last_fullsized":"2024-03-16T10:06:09.929293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:46:07.305218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.598761+0000","last_change":"2024-03-16T10:06:03.227303+0000","last_active":"2024-03-16T10:06:40.598761+0000","last_peered":"2024-03-16T10:06:40.598761+0000","last_clean":"2024-03-16T10:06:40.598761+0000","last_became_active":"2024-03-16T10:06:03.226953+0000","last_became_peered":"2024-03-16T10:06:03.226953+0000","last_unstale":"2024-03-16T10:06:40.598761+0000","last_undegraded":"2024-03-16T10:06:40.598761+0000","last_fullsized":"2024-03-16T10:06:40.598761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:28:40.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333170+0000","last_change":"2024-03-16T10:06:04.220502+0000","last_active":"2024-03-16T10:06:09.333170+0000","last_peered":"2024-03-16T10:06:09.333170+0000","last_clean":"2024-03-16T10:06:09.333170+0000","last_became_active":"2024-03-16T10:06:04.220057+0000","last_became_peered":"2024-03-16T10:06:04.220057+0000","last_unstale":"2024-03-16T10:06:09.333170+0000","last_undegraded":"2024-03-16T10:06:09.333170+0000","last_fullsized":"2024-03-16T10:06:09.333170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:46:30.972382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929131+0000","last_change":"2024-03-16T10:06:08.346548+0000","last_active":"2024-03-16T10:06:09.929131+0000","last_peered":"2024-03-16T10:06:09.929131+0000","last_clean":"2024-03-16T10:06:09.929131+0000","last_became_active":"2024-03-16T10:06:08.346145+0000","last_became_peered":"2024-03-16T10:06:08.346145+0000","last_unstale":"2024-03-16T10:06:09.929131+0000","last_undegraded":"2024-03-16T10:06:09.929131+0000","last_fullsized":"2024-03-16T10:06:09.929131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:49.221773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.569909+0000","last_change":"2024-03-16T10:06:03.222623+0000","last_active":"2024-03-16T10:06:39.569909+0000","last_peered":"2024-03-16T10:06:39.569909+0000","last_clean":"2024-03-16T10:06:39.569909+0000","last_became_active":"2024-03-16T10:06:03.222300+0000","last_became_peered":"2024-03-16T10:06:03.222300+0000","last_unstale":"2024-03-16T10:06:39.569909+0000","last_undegraded":"2024-03-16T10:06:39.569909+0000","last_fullsized":"2024-03-16T10:06:39.569909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:49.397327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333607+0000","last_change":"2024-03-16T10:06:04.194004+0000","last_active":"2024-03-16T10:06:09.333607+0000","last_peered":"2024-03-16T10:06:09.333607+0000","last_clean":"2024-03-16T10:06:09.333607+0000","last_became_active":"2024-03-16T10:06:04.193551+0000","last_became_peered":"2024-03-16T10:06:04.193551+0000","last_unstale":"2024-03-16T10:06:09.333607+0000","last_undegraded":"2024-03-16T10:06:09.333607+0000","last_fullsized":"2024-03-16T10:06:09.333607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:59:25.668511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930255+0000","last_change":"2024-03-16T10:06:08.330170+0000","last_active":"2024-03-16T10:06:09.930255+0000","last_peered":"2024-03-16T10:06:09.930255+0000","last_clean":"2024-03-16T10:06:09.930255+0000","last_became_active":"2024-03-16T10:06:08.329699+0000","last_became_peered":"2024-03-16T10:06:08.329699+0000","last_unstale":"2024-03-16T10:06:09.930255+0000","last_undegraded":"2024-03-16T10:06:09.930255+0000","last_fullsized":"2024-03-16T10:06:09.930255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:50.571845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.518523+0000","last_change":"2024-03-16T10:06:03.228039+0000","last_active":"2024-03-16T10:06:36.518523+0000","last_peered":"2024-03-16T10:06:36.518523+0000","last_clean":"2024-03-16T10:06:36.518523+0000","last_became_active":"2024-03-16T10:06:03.227715+0000","last_became_peered":"2024-03-16T10:06:03.227715+0000","last_unstale":"2024-03-16T10:06:36.518523+0000","last_undegraded":"2024-03-16T10:06:36.518523+0000","last_fullsized":"2024-03-16T10:06:36.518523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:13:08.980781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931051+0000","last_change":"2024-03-16T10:06:04.214274+0000","last_active":"2024-03-16T10:06:09.931051+0000","last_peered":"2024-03-16T10:06:09.931051+0000","last_clean":"2024-03-16T10:06:09.931051+0000","last_became_active":"2024-03-16T10:06:04.213979+0000","last_became_peered":"2024-03-16T10:06:04.213979+0000","last_unstale":"2024-03-16T10:06:09.931051+0000","last_undegraded":"2024-03-16T10:06:09.931051+0000","last_fullsized":"2024-03-16T10:06:09.931051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:12.684609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930021+0000","last_change":"2024-03-16T10:06:08.346874+0000","last_active":"2024-03-16T10:06:09.930021+0000","last_peered":"2024-03-16T10:06:09.930021+0000","last_clean":"2024-03-16T10:06:09.930021+0000","last_became_active":"2024-03-16T10:06:08.346450+0000","last_became_peered":"2024-03-16T10:06:08.346450+0000","last_unstale":"2024-03-16T10:06:09.930021+0000","last_undegraded":"2024-03-16T10:06:09.930021+0000","last_fullsized":"2024-03-16T10:06:09.930021+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:09:12.159483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.556116+0000","last_change":"2024-03-16T10:06:03.236088+0000","last_active":"2024-03-16T10:06:38.556116+0000","last_peered":"2024-03-16T10:06:38.556116+0000","last_clean":"2024-03-16T10:06:38.556116+0000","last_became_active":"2024-03-16T10:06:03.235768+0000","last_became_peered":"2024-03-16T10:06:03.235768+0000","last_unstale":"2024-03-16T10:06:38.556116+0000","last_undegraded":"2024-03-16T10:06:38.556116+0000","last_fullsized":"2024-03-16T10:06:38.556116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:57:38.813913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333976+0000","last_change":"2024-03-16T10:06:04.224331+0000","last_active":"2024-03-16T10:06:09.333976+0000","last_peered":"2024-03-16T10:06:09.333976+0000","last_clean":"2024-03-16T10:06:09.333976+0000","last_became_active":"2024-03-16T10:06:04.223946+0000","last_became_peered":"2024-03-16T10:06:04.223946+0000","last_unstale":"2024-03-16T10:06:09.333976+0000","last_undegraded":"2024-03-16T10:06:09.333976+0000","last_fullsized":"2024-03-16T10:06:09.333976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:58:56.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930996+0000","last_change":"2024-03-16T10:06:08.344757+0000","last_active":"2024-03-16T10:06:09.930996+0000","last_peered":"2024-03-16T10:06:09.930996+0000","last_clean":"2024-03-16T10:06:09.930996+0000","last_became_active":"2024-03-16T10:06:08.344275+0000","last_became_peered":"2024-03-16T10:06:08.344275+0000","last_unstale":"2024-03-16T10:06:09.930996+0000","last_undegraded":"2024-03-16T10:06:09.930996+0000","last_fullsized":"2024-03-16T10:06:09.930996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:31:18.545734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:37.539735+0000","last_change":"2024-03-16T10:06:03.239619+0000","last_active":"2024-03-16T10:06:37.539735+0000","last_peered":"2024-03-16T10:06:37.539735+0000","last_clean":"2024-03-16T10:06:37.539735+0000","last_became_active":"2024-03-16T10:06:03.238982+0000","last_became_peered":"2024-03-16T10:06:03.238982+0000","last_unstale":"2024-03-16T10:06:37.539735+0000","last_undegraded":"2024-03-16T10:06:37.539735+0000","last_fullsized":"2024-03-16T10:06:37.539735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:20.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415080+0000","last_change":"2024-03-16T10:06:04.221897+0000","last_active":"2024-03-16T10:06:09.415080+0000","last_peered":"2024-03-16T10:06:09.415080+0000","last_clean":"2024-03-16T10:06:09.415080+0000","last_became_active":"2024-03-16T10:06:04.218243+0000","last_became_peered":"2024-03-16T10:06:04.218243+0000","last_unstale":"2024-03-16T10:06:09.415080+0000","last_undegraded":"2024-03-16T10:06:09.415080+0000","last_fullsized":"2024-03-16T10:06:09.415080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:25.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337260+0000","last_change":"2024-03-16T10:06:08.352722+0000","last_active":"2024-03-16T10:06:09.337260+0000","last_peered":"2024-03-16T10:06:09.337260+0000","last_clean":"2024-03-16T10:06:09.337260+0000","last_became_active":"2024-03-16T10:06:08.352292+0000","last_became_peered":"2024-03-16T10:06:08.352292+0000","last_unstale":"2024-03-16T10:06:09.337260+0000","last_undegraded":"2024-03-16T10:06:09.337260+0000","last_fullsized":"2024-03-16T10:06:09.337260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:56:23.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333255+0000","last_change":"2024-03-16T10:06:03.224143+0000","last_active":"2024-03-16T10:06:09.333255+0000","last_peered":"2024-03-16T10:06:09.333255+0000","last_clean":"2024-03-16T10:06:09.333255+0000","last_became_active":"2024-03-16T10:06:03.223810+0000","last_became_peered":"2024-03-16T10:06:03.223810+0000","last_unstale":"2024-03-16T10:06:09.333255+0000","last_undegraded":"2024-03-16T10:06:09.333255+0000","last_fullsized":"2024-03-16T10:06:09.333255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:26.724292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930872+0000","last_change":"2024-03-16T10:06:04.232758+0000","last_active":"2024-03-16T10:06:09.930872+0000","last_peered":"2024-03-16T10:06:09.930872+0000","last_clean":"2024-03-16T10:06:09.930872+0000","last_became_active":"2024-03-16T10:06:04.232371+0000","last_became_peered":"2024-03-16T10:06:04.232371+0000","last_unstale":"2024-03-16T10:06:09.930872+0000","last_undegraded":"2024-03-16T10:06:09.930872+0000","last_fullsized":"2024-03-16T10:06:09.930872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:00.527708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415720+0000","last_change":"2024-03-16T10:06:08.345360+0000","last_active":"2024-03-16T10:06:09.415720+0000","last_peered":"2024-03-16T10:06:09.415720+0000","last_clean":"2024-03-16T10:06:09.415720+0000","last_became_active":"2024-03-16T10:06:08.336245+0000","last_became_peered":"2024-03-16T10:06:08.336245+0000","last_unstale":"2024-03-16T10:06:09.415720+0000","last_undegraded":"2024-03-16T10:06:09.415720+0000","last_fullsized":"2024-03-16T10:06:09.415720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:14:33.324555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:41.600709+0000","last_change":"2024-03-16T10:06:03.237015+0000","last_active":"2024-03-16T10:06:41.600709+0000","last_peered":"2024-03-16T10:06:41.600709+0000","last_clean":"2024-03-16T10:06:41.600709+0000","last_became_active":"2024-03-16T10:06:03.236722+0000","last_became_peered":"2024-03-16T10:06:03.236722+0000","last_unstale":"2024-03-16T10:06:41.600709+0000","last_undegraded":"2024-03-16T10:06:41.600709+0000","last_fullsized":"2024-03-16T10:06:41.600709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:10:03.403854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333627+0000","last_change":"2024-03-16T10:06:04.209017+0000","last_active":"2024-03-16T10:06:09.333627+0000","last_peered":"2024-03-16T10:06:09.333627+0000","last_clean":"2024-03-16T10:06:09.333627+0000","last_became_active":"2024-03-16T10:06:04.208665+0000","last_became_peered":"2024-03-16T10:06:04.208665+0000","last_unstale":"2024-03-16T10:06:09.333627+0000","last_undegraded":"2024-03-16T10:06:09.333627+0000","last_fullsized":"2024-03-16T10:06:09.333627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:27.074480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931172+0000","last_change":"2024-03-16T10:06:08.340510+0000","last_active":"2024-03-16T10:06:09.931172+0000","last_peered":"2024-03-16T10:06:09.931172+0000","last_clean":"2024-03-16T10:06:09.931172+0000","last_became_active":"2024-03-16T10:06:08.340070+0000","last_became_peered":"2024-03-16T10:06:08.340070+0000","last_unstale":"2024-03-16T10:06:09.931172+0000","last_undegraded":"2024-03-16T10:06:09.931172+0000","last_fullsized":"2024-03-16T10:06:09.931172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:33:55.038855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.569218+0000","last_change":"2024-03-16T10:06:03.225770+0000","last_active":"2024-03-16T10:06:39.569218+0000","last_peered":"2024-03-16T10:06:39.569218+0000","last_clean":"2024-03-16T10:06:39.569218+0000","last_became_active":"2024-03-16T10:06:03.223734+0000","last_became_peered":"2024-03-16T10:06:03.223734+0000","last_unstale":"2024-03-16T10:06:39.569218+0000","last_undegraded":"2024-03-16T10:06:39.569218+0000","last_fullsized":"2024-03-16T10:06:39.569218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:13.764147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337450+0000","last_change":"2024-03-16T10:06:04.206027+0000","last_active":"2024-03-16T10:06:09.337450+0000","last_peered":"2024-03-16T10:06:09.337450+0000","last_clean":"2024-03-16T10:06:09.337450+0000","last_became_active":"2024-03-16T10:06:04.191657+0000","last_became_peered":"2024-03-16T10:06:04.191657+0000","last_unstale":"2024-03-16T10:06:09.337450+0000","last_undegraded":"2024-03-16T10:06:09.337450+0000","last_fullsized":"2024-03-16T10:06:09.337450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:52.839646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929615+0000","last_change":"2024-03-16T10:06:08.345595+0000","last_active":"2024-03-16T10:06:09.929615+0000","last_peered":"2024-03-16T10:06:09.929615+0000","last_clean":"2024-03-16T10:06:09.929615+0000","last_became_active":"2024-03-16T10:06:08.345205+0000","last_became_peered":"2024-03-16T10:06:08.345205+0000","last_unstale":"2024-03-16T10:06:09.929615+0000","last_undegraded":"2024-03-16T10:06:09.929615+0000","last_fullsized":"2024-03-16T10:06:09.929615+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:37:51.067056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333485+0000","last_change":"2024-03-16T10:06:03.219716+0000","last_active":"2024-03-16T10:06:09.333485+0000","last_peered":"2024-03-16T10:06:09.333485+0000","last_clean":"2024-03-16T10:06:09.333485+0000","last_became_active":"2024-03-16T10:06:03.219375+0000","last_became_peered":"2024-03-16T10:06:03.219375+0000","last_unstale":"2024-03-16T10:06:09.333485+0000","last_undegraded":"2024-03-16T10:06:09.333485+0000","last_fullsized":"2024-03-16T10:06:09.333485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:23:51.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333378+0000","last_change":"2024-03-16T10:06:04.193605+0000","last_active":"2024-03-16T10:06:09.333378+0000","last_peered":"2024-03-16T10:06:09.333378+0000","last_clean":"2024-03-16T10:06:09.333378+0000","last_became_active":"2024-03-16T10:06:04.193295+0000","last_became_peered":"2024-03-16T10:06:04.193295+0000","last_unstale":"2024-03-16T10:06:09.333378+0000","last_undegraded":"2024-03-16T10:06:09.333378+0000","last_fullsized":"2024-03-16T10:06:09.333378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:10:04.140279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931302+0000","last_change":"2024-03-16T10:06:08.338862+0000","last_active":"2024-03-16T10:06:09.931302+0000","last_peered":"2024-03-16T10:06:09.931302+0000","last_clean":"2024-03-16T10:06:09.931302+0000","last_became_active":"2024-03-16T10:06:08.337221+0000","last_became_peered":"2024-03-16T10:06:08.337221+0000","last_unstale":"2024-03-16T10:06:09.931302+0000","last_undegraded":"2024-03-16T10:06:09.931302+0000","last_fullsized":"2024-03-16T10:06:09.931302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:28:51.251299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:41.606270+0000","last_change":"2024-03-16T10:06:03.220555+0000","last_active":"2024-03-16T10:06:41.606270+0000","last_peered":"2024-03-16T10:06:41.606270+0000","last_clean":"2024-03-16T10:06:41.606270+0000","last_became_active":"2024-03-16T10:06:03.220224+0000","last_became_peered":"2024-03-16T10:06:03.220224+0000","last_unstale":"2024-03-16T10:06:41.606270+0000","last_undegraded":"2024-03-16T10:06:41.606270+0000","last_fullsized":"2024-03-16T10:06:41.606270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:12:47.568132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":44,"num_write":28,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931288+0000","last_change":"2024-03-16T10:06:04.195957+0000","last_active":"2024-03-16T10:06:09.931288+0000","last_peered":"2024-03-16T10:06:09.931288+0000","last_clean":"2024-03-16T10:06:09.931288+0000","last_became_active":"2024-03-16T10:06:04.195539+0000","last_became_peered":"2024-03-16T10:06:04.195539+0000","last_unstale":"2024-03-16T10:06:09.931288+0000","last_undegraded":"2024-03-16T10:06:09.931288+0000","last_fullsized":"2024-03-16T10:06:09.931288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:39:45.035936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.334156+0000","last_change":"2024-03-16T10:06:08.347941+0000","last_active":"2024-03-16T10:06:09.334156+0000","last_peered":"2024-03-16T10:06:09.334156+0000","last_clean":"2024-03-16T10:06:09.334156+0000","last_became_active":"2024-03-16T10:06:08.347517+0000","last_became_peered":"2024-03-16T10:06:08.347517+0000","last_unstale":"2024-03-16T10:06:09.334156+0000","last_undegraded":"2024-03-16T10:06:09.334156+0000","last_fullsized":"2024-03-16T10:06:09.334156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:51:08.696206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.598434+0000","last_change":"2024-03-16T10:06:03.235652+0000","last_active":"2024-03-16T10:06:40.598434+0000","last_peered":"2024-03-16T10:06:40.598434+0000","last_clean":"2024-03-16T10:06:40.598434+0000","last_became_active":"2024-03-16T10:06:03.235219+0000","last_became_peered":"2024-03-16T10:06:03.235219+0000","last_unstale":"2024-03-16T10:06:40.598434+0000","last_undegraded":"2024-03-16T10:06:40.598434+0000","last_fullsized":"2024-03-16T10:06:40.598434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:23.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931581+0000","last_change":"2024-03-16T10:06:04.225460+0000","last_active":"2024-03-16T10:06:09.931581+0000","last_peered":"2024-03-16T10:06:09.931581+0000","last_clean":"2024-03-16T10:06:09.931581+0000","last_became_active":"2024-03-16T10:06:04.225019+0000","last_became_peered":"2024-03-16T10:06:04.225019+0000","last_unstale":"2024-03-16T10:06:09.931581+0000","last_undegraded":"2024-03-16T10:06:09.931581+0000","last_fullsized":"2024-03-16T10:06:09.931581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:42:06.525298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930960+0000","last_change":"2024-03-16T10:06:08.344775+0000","last_active":"2024-03-16T10:06:09.930960+0000","last_peered":"2024-03-16T10:06:09.930960+0000","last_clean":"2024-03-16T10:06:09.930960+0000","last_became_active":"2024-03-16T10:06:08.344360+0000","last_became_peered":"2024-03-16T10:06:08.344360+0000","last_unstale":"2024-03-16T10:06:09.930960+0000","last_undegraded":"2024-03-16T10:06:09.930960+0000","last_fullsized":"2024-03-16T10:06:09.930960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:26.710955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.519873+0000","last_change":"2024-03-16T10:06:03.236887+0000","last_active":"2024-03-16T10:06:36.519873+0000","last_peered":"2024-03-16T10:06:36.519873+0000","last_clean":"2024-03-16T10:06:36.519873+0000","last_became_active":"2024-03-16T10:06:03.236454+0000","last_became_peered":"2024-03-16T10:06:03.236454+0000","last_unstale":"2024-03-16T10:06:36.519873+0000","last_undegraded":"2024-03-16T10:06:36.519873+0000","last_fullsized":"2024-03-16T10:06:36.519873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:03.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599575+0000","last_change":"2024-03-16T10:06:04.220008+0000","last_active":"2024-03-16T10:06:09.599575+0000","last_peered":"2024-03-16T10:06:09.599575+0000","last_clean":"2024-03-16T10:06:09.599575+0000","last_became_active":"2024-03-16T10:06:04.219683+0000","last_became_peered":"2024-03-16T10:06:04.219683+0000","last_unstale":"2024-03-16T10:06:09.599575+0000","last_undegraded":"2024-03-16T10:06:09.599575+0000","last_fullsized":"2024-03-16T10:06:09.599575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:35.577474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931338+0000","last_change":"2024-03-16T10:06:08.352793+0000","last_active":"2024-03-16T10:06:09.931338+0000","last_peered":"2024-03-16T10:06:09.931338+0000","last_clean":"2024-03-16T10:06:09.931338+0000","last_became_active":"2024-03-16T10:06:08.352272+0000","last_became_peered":"2024-03-16T10:06:08.352272+0000","last_unstale":"2024-03-16T10:06:09.931338+0000","last_undegraded":"2024-03-16T10:06:09.931338+0000","last_fullsized":"2024-03-16T10:06:09.931338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:39.835078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.550722+0000","last_change":"2024-03-16T10:06:03.224301+0000","last_active":"2024-03-16T10:06:38.550722+0000","last_peered":"2024-03-16T10:06:38.550722+0000","last_clean":"2024-03-16T10:06:38.550722+0000","last_became_active":"2024-03-16T10:06:03.223899+0000","last_became_peered":"2024-03-16T10:06:03.223899+0000","last_unstale":"2024-03-16T10:06:38.550722+0000","last_undegraded":"2024-03-16T10:06:38.550722+0000","last_fullsized":"2024-03-16T10:06:38.550722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:39.244106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599346+0000","last_change":"2024-03-16T10:06:04.193649+0000","last_active":"2024-03-16T10:06:09.599346+0000","last_peered":"2024-03-16T10:06:09.599346+0000","last_clean":"2024-03-16T10:06:09.599346+0000","last_became_active":"2024-03-16T10:06:04.193275+0000","last_became_peered":"2024-03-16T10:06:04.193275+0000","last_unstale":"2024-03-16T10:06:09.599346+0000","last_undegraded":"2024-03-16T10:06:09.599346+0000","last_fullsized":"2024-03-16T10:06:09.599346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:59.597769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932892+0000","last_change":"2024-03-16T10:06:08.349918+0000","last_active":"2024-03-16T10:06:09.932892+0000","last_peered":"2024-03-16T10:06:09.932892+0000","last_clean":"2024-03-16T10:06:09.932892+0000","last_became_active":"2024-03-16T10:06:08.349506+0000","last_became_peered":"2024-03-16T10:06:08.349506+0000","last_unstale":"2024-03-16T10:06:09.932892+0000","last_undegraded":"2024-03-16T10:06:09.932892+0000","last_fullsized":"2024-03-16T10:06:09.932892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:08.146594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.568645+0000","last_change":"2024-03-16T10:06:03.231888+0000","last_active":"2024-03-16T10:06:39.568645+0000","last_peered":"2024-03-16T10:06:39.568645+0000","last_clean":"2024-03-16T10:06:39.568645+0000","last_became_active":"2024-03-16T10:06:03.231576+0000","last_became_peered":"2024-03-16T10:06:03.231576+0000","last_unstale":"2024-03-16T10:06:39.568645+0000","last_undegraded":"2024-03-16T10:06:39.568645+0000","last_fullsized":"2024-03-16T10:06:39.568645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:46:48.080754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333183+0000","last_change":"2024-03-16T10:06:04.194629+0000","last_active":"2024-03-16T10:06:09.333183+0000","last_peered":"2024-03-16T10:06:09.333183+0000","last_clean":"2024-03-16T10:06:09.333183+0000","last_became_active":"2024-03-16T10:06:04.194306+0000","last_became_peered":"2024-03-16T10:06:04.194306+0000","last_unstale":"2024-03-16T10:06:09.333183+0000","last_undegraded":"2024-03-16T10:06:09.333183+0000","last_fullsized":"2024-03-16T10:06:09.333183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:36.455569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333244+0000","last_change":"2024-03-16T10:06:08.349269+0000","last_active":"2024-03-16T10:06:09.333244+0000","last_peered":"2024-03-16T10:06:09.333244+0000","last_clean":"2024-03-16T10:06:09.333244+0000","last_became_active":"2024-03-16T10:06:08.348824+0000","last_became_peered":"2024-03-16T10:06:08.348824+0000","last_unstale":"2024-03-16T10:06:09.333244+0000","last_undegraded":"2024-03-16T10:06:09.333244+0000","last_fullsized":"2024-03-16T10:06:09.333244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:26.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.532429+0000","last_change":"2024-03-16T10:06:03.221066+0000","last_active":"2024-03-16T10:06:39.532429+0000","last_peered":"2024-03-16T10:06:39.532429+0000","last_clean":"2024-03-16T10:06:39.532429+0000","last_became_active":"2024-03-16T10:06:03.219908+0000","last_became_peered":"2024-03-16T10:06:03.219908+0000","last_unstale":"2024-03-16T10:06:39.532429+0000","last_undegraded":"2024-03-16T10:06:39.532429+0000","last_fullsized":"2024-03-16T10:06:39.532429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:41.026293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930634+0000","last_change":"2024-03-16T10:06:04.198752+0000","last_active":"2024-03-16T10:06:09.930634+0000","last_peered":"2024-03-16T10:06:09.930634+0000","last_clean":"2024-03-16T10:06:09.930634+0000","last_became_active":"2024-03-16T10:06:04.198016+0000","last_became_peered":"2024-03-16T10:06:04.198016+0000","last_unstale":"2024-03-16T10:06:09.930634+0000","last_undegraded":"2024-03-16T10:06:09.930634+0000","last_fullsized":"2024-03-16T10:06:09.930634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:48:09.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930986+0000","last_change":"2024-03-16T10:06:08.529441+0000","last_active":"2024-03-16T10:06:09.930986+0000","last_peered":"2024-03-16T10:06:09.930986+0000","last_clean":"2024-03-16T10:06:09.930986+0000","last_became_active":"2024-03-16T10:06:08.528801+0000","last_became_peered":"2024-03-16T10:06:08.528801+0000","last_unstale":"2024-03-16T10:06:09.930986+0000","last_undegraded":"2024-03-16T10:06:09.930986+0000","last_fullsized":"2024-03-16T10:06:09.930986+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:50.592415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931113+0000","last_change":"2024-03-16T10:06:03.236260+0000","last_active":"2024-03-16T10:06:09.931113+0000","last_peered":"2024-03-16T10:06:09.931113+0000","last_clean":"2024-03-16T10:06:09.931113+0000","last_became_active":"2024-03-16T10:06:03.235792+0000","last_became_peered":"2024-03-16T10:06:03.235792+0000","last_unstale":"2024-03-16T10:06:09.931113+0000","last_undegraded":"2024-03-16T10:06:09.931113+0000","last_fullsized":"2024-03-16T10:06:09.931113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:11:59.364244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930244+0000","last_change":"2024-03-16T10:06:04.226962+0000","last_active":"2024-03-16T10:06:09.930244+0000","last_peered":"2024-03-16T10:06:09.930244+0000","last_clean":"2024-03-16T10:06:09.930244+0000","last_became_active":"2024-03-16T10:06:04.226668+0000","last_became_peered":"2024-03-16T10:06:04.226668+0000","last_unstale":"2024-03-16T10:06:09.930244+0000","last_undegraded":"2024-03-16T10:06:09.930244+0000","last_fullsized":"2024-03-16T10:06:09.930244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:53.097834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932114+0000","last_change":"2024-03-16T10:06:08.343528+0000","last_active":"2024-03-16T10:06:09.932114+0000","last_peered":"2024-03-16T10:06:09.932114+0000","last_clean":"2024-03-16T10:06:09.932114+0000","last_became_active":"2024-03-16T10:06:08.340969+0000","last_became_peered":"2024-03-16T10:06:08.340969+0000","last_unstale":"2024-03-16T10:06:09.932114+0000","last_undegraded":"2024-03-16T10:06:09.932114+0000","last_fullsized":"2024-03-16T10:06:09.932114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:54:33.028756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.602835+0000","last_change":"2024-03-16T10:06:03.236166+0000","last_active":"2024-03-16T10:06:40.602835+0000","last_peered":"2024-03-16T10:06:40.602835+0000","last_clean":"2024-03-16T10:06:40.602835+0000","last_became_active":"2024-03-16T10:06:03.235825+0000","last_became_peered":"2024-03-16T10:06:03.235825+0000","last_unstale":"2024-03-16T10:06:40.602835+0000","last_undegraded":"2024-03-16T10:06:40.602835+0000","last_fullsized":"2024-03-16T10:06:40.602835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:39:34.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929916+0000","last_change":"2024-03-16T10:06:04.220002+0000","last_active":"2024-03-16T10:06:09.929916+0000","last_peered":"2024-03-16T10:06:09.929916+0000","last_clean":"2024-03-16T10:06:09.929916+0000","last_became_active":"2024-03-16T10:06:04.219623+0000","last_became_peered":"2024-03-16T10:06:04.219623+0000","last_unstale":"2024-03-16T10:06:09.929916+0000","last_undegraded":"2024-03-16T10:06:09.929916+0000","last_fullsized":"2024-03-16T10:06:09.929916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:39:35.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932650+0000","last_change":"2024-03-16T10:06:08.530396+0000","last_active":"2024-03-16T10:06:09.932650+0000","last_peered":"2024-03-16T10:06:09.932650+0000","last_clean":"2024-03-16T10:06:09.932650+0000","last_became_active":"2024-03-16T10:06:08.529102+0000","last_became_peered":"2024-03-16T10:06:08.529102+0000","last_unstale":"2024-03-16T10:06:09.932650+0000","last_undegraded":"2024-03-16T10:06:09.932650+0000","last_fullsized":"2024-03-16T10:06:09.932650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:36.408884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333576+0000","last_change":"2024-03-16T10:06:04.193150+0000","last_active":"2024-03-16T10:06:09.333576+0000","last_peered":"2024-03-16T10:06:09.333576+0000","last_clean":"2024-03-16T10:06:09.333576+0000","last_became_active":"2024-03-16T10:06:04.192847+0000","last_became_peered":"2024-03-16T10:06:04.192847+0000","last_unstale":"2024-03-16T10:06:09.333576+0000","last_undegraded":"2024-03-16T10:06:09.333576+0000","last_fullsized":"2024-03-16T10:06:09.333576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:31:22.687107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.600754+0000","last_change":"2024-03-16T10:06:03.247109+0000","last_active":"2024-03-16T10:06:40.600754+0000","last_peered":"2024-03-16T10:06:40.600754+0000","last_clean":"2024-03-16T10:06:40.600754+0000","last_became_active":"2024-03-16T10:06:03.246515+0000","last_became_peered":"2024-03-16T10:06:03.246515+0000","last_unstale":"2024-03-16T10:06:40.600754+0000","last_undegraded":"2024-03-16T10:06:40.600754+0000","last_fullsized":"2024-03-16T10:06:40.600754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:29.744828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:11.723238+0000","last_change":"2024-03-16T10:06:08.334587+0000","last_active":"2024-03-16T10:06:11.723238+0000","last_peered":"2024-03-16T10:06:11.723238+0000","last_clean":"2024-03-16T10:06:11.723238+0000","last_became_active":"2024-03-16T10:06:08.334248+0000","last_became_peered":"2024-03-16T10:06:08.334248+0000","last_unstale":"2024-03-16T10:06:11.723238+0000","last_undegraded":"2024-03-16T10:06:11.723238+0000","last_fullsized":"2024-03-16T10:06:11.723238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:33:28.909540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415334+0000","last_change":"2024-03-16T10:06:04.221069+0000","last_active":"2024-03-16T10:06:09.415334+0000","last_peered":"2024-03-16T10:06:09.415334+0000","last_clean":"2024-03-16T10:06:09.415334+0000","last_became_active":"2024-03-16T10:06:04.218197+0000","last_became_peered":"2024-03-16T10:06:04.218197+0000","last_unstale":"2024-03-16T10:06:09.415334+0000","last_undegraded":"2024-03-16T10:06:09.415334+0000","last_fullsized":"2024-03-16T10:06:09.415334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:38:25.262182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.552124+0000","last_change":"2024-03-16T10:06:03.221284+0000","last_active":"2024-03-16T10:06:38.552124+0000","last_peered":"2024-03-16T10:06:38.552124+0000","last_clean":"2024-03-16T10:06:38.552124+0000","last_became_active":"2024-03-16T10:06:03.220467+0000","last_became_peered":"2024-03-16T10:06:03.220467+0000","last_unstale":"2024-03-16T10:06:38.552124+0000","last_undegraded":"2024-03-16T10:06:38.552124+0000","last_fullsized":"2024-03-16T10:06:38.552124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:31:19.753996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599230+0000","last_change":"2024-03-16T10:04:45.615656+0000","last_active":"2024-03-16T10:06:09.599230+0000","last_peered":"2024-03-16T10:06:09.599230+0000","last_clean":"2024-03-16T10:06:09.599230+0000","last_became_active":"2024-03-16T10:04:45.615261+0000","last_became_peered":"2024-03-16T10:04:45.615261+0000","last_unstale":"2024-03-16T10:06:09.599230+0000","last_undegraded":"2024-03-16T10:06:09.599230+0000","last_fullsized":"2024-03-16T10:06:09.599230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_clean_scrub_stamp":"2024-03-16T10:00:56.880115+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:06.546840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333566+0000","last_change":"2024-03-16T10:06:08.529863+0000","last_active":"2024-03-16T10:06:09.333566+0000","last_peered":"2024-03-16T10:06:09.333566+0000","last_clean":"2024-03-16T10:06:09.333566+0000","last_became_active":"2024-03-16T10:06:08.528934+0000","last_became_peered":"2024-03-16T10:06:08.528934+0000","last_unstale":"2024-03-16T10:06:09.333566+0000","last_undegraded":"2024-03-16T10:06:09.333566+0000","last_fullsized":"2024-03-16T10:06:09.333566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:40.443983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930375+0000","last_change":"2024-03-16T10:06:04.206776+0000","last_active":"2024-03-16T10:06:09.930375+0000","last_peered":"2024-03-16T10:06:09.930375+0000","last_clean":"2024-03-16T10:06:09.930375+0000","last_became_active":"2024-03-16T10:06:04.206333+0000","last_became_peered":"2024-03-16T10:06:04.206333+0000","last_unstale":"2024-03-16T10:06:09.930375+0000","last_undegraded":"2024-03-16T10:06:09.930375+0000","last_fullsized":"2024-03-16T10:06:09.930375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:00:34.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:37.536579+0000","last_change":"2024-03-16T10:06:03.226921+0000","last_active":"2024-03-16T10:06:37.536579+0000","last_peered":"2024-03-16T10:06:37.536579+0000","last_clean":"2024-03-16T10:06:37.536579+0000","last_became_active":"2024-03-16T10:06:03.226596+0000","last_became_peered":"2024-03-16T10:06:03.226596+0000","last_unstale":"2024-03-16T10:06:37.536579+0000","last_undegraded":"2024-03-16T10:06:37.536579+0000","last_fullsized":"2024-03-16T10:06:37.536579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:03:50.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930497+0000","last_change":"2024-03-16T10:06:08.344432+0000","last_active":"2024-03-16T10:06:09.930497+0000","last_peered":"2024-03-16T10:06:09.930497+0000","last_clean":"2024-03-16T10:06:09.930497+0000","last_became_active":"2024-03-16T10:06:08.343983+0000","last_became_peered":"2024-03-16T10:06:08.343983+0000","last_unstale":"2024-03-16T10:06:09.930497+0000","last_undegraded":"2024-03-16T10:06:09.930497+0000","last_fullsized":"2024-03-16T10:06:09.930497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:41.686700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.573568+0000","last_change":"2024-03-16T10:06:03.232806+0000","last_active":"2024-03-16T10:06:39.573568+0000","last_peered":"2024-03-16T10:06:39.573568+0000","last_clean":"2024-03-16T10:06:39.573568+0000","last_became_active":"2024-03-16T10:06:03.232465+0000","last_became_peered":"2024-03-16T10:06:03.232465+0000","last_unstale":"2024-03-16T10:06:39.573568+0000","last_undegraded":"2024-03-16T10:06:39.573568+0000","last_fullsized":"2024-03-16T10:06:39.573568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:36.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932036+0000","last_change":"2024-03-16T10:06:04.215899+0000","last_active":"2024-03-16T10:06:09.932036+0000","last_peered":"2024-03-16T10:06:09.932036+0000","last_clean":"2024-03-16T10:06:09.932036+0000","last_became_active":"2024-03-16T10:06:04.215474+0000","last_became_peered":"2024-03-16T10:06:04.215474+0000","last_unstale":"2024-03-16T10:06:09.932036+0000","last_undegraded":"2024-03-16T10:06:09.932036+0000","last_fullsized":"2024-03-16T10:06:09.932036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:27:26.663702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415230+0000","last_change":"2024-03-16T10:06:08.530852+0000","last_active":"2024-03-16T10:06:09.415230+0000","last_peered":"2024-03-16T10:06:09.415230+0000","last_clean":"2024-03-16T10:06:09.415230+0000","last_became_active":"2024-03-16T10:06:08.529386+0000","last_became_peered":"2024-03-16T10:06:08.529386+0000","last_unstale":"2024-03-16T10:06:09.415230+0000","last_undegraded":"2024-03-16T10:06:09.415230+0000","last_fullsized":"2024-03-16T10:06:09.415230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:37:17.201323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.238469+0000","last_change":"2024-03-16T10:06:03.244012+0000","last_active":"2024-03-16T10:06:40.238469+0000","last_peered":"2024-03-16T10:06:40.238469+0000","last_clean":"2024-03-16T10:06:40.238469+0000","last_became_active":"2024-03-16T10:06:03.243612+0000","last_became_peered":"2024-03-16T10:06:03.243612+0000","last_unstale":"2024-03-16T10:06:40.238469+0000","last_undegraded":"2024-03-16T10:06:40.238469+0000","last_fullsized":"2024-03-16T10:06:40.238469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:32:34.807111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416162+0000","last_change":"2024-03-16T10:06:04.226678+0000","last_active":"2024-03-16T10:06:09.416162+0000","last_peered":"2024-03-16T10:06:09.416162+0000","last_clean":"2024-03-16T10:06:09.416162+0000","last_became_active":"2024-03-16T10:06:04.226214+0000","last_became_peered":"2024-03-16T10:06:04.226214+0000","last_unstale":"2024-03-16T10:06:09.416162+0000","last_undegraded":"2024-03-16T10:06:09.416162+0000","last_fullsized":"2024-03-16T10:06:09.416162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:05.732310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929744+0000","last_change":"2024-03-16T10:06:08.326985+0000","last_active":"2024-03-16T10:06:09.929744+0000","last_peered":"2024-03-16T10:06:09.929744+0000","last_clean":"2024-03-16T10:06:09.929744+0000","last_became_active":"2024-03-16T10:06:08.326429+0000","last_became_peered":"2024-03-16T10:06:08.326429+0000","last_unstale":"2024-03-16T10:06:09.929744+0000","last_undegraded":"2024-03-16T10:06:09.929744+0000","last_fullsized":"2024-03-16T10:06:09.929744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:44:59.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.515608+0000","last_change":"2024-03-16T10:06:03.232910+0000","last_active":"2024-03-16T10:06:36.515608+0000","last_peered":"2024-03-16T10:06:36.515608+0000","last_clean":"2024-03-16T10:06:36.515608+0000","last_became_active":"2024-03-16T10:06:03.232682+0000","last_became_peered":"2024-03-16T10:06:03.232682+0000","last_unstale":"2024-03-16T10:06:36.515608+0000","last_undegraded":"2024-03-16T10:06:36.515608+0000","last_fullsized":"2024-03-16T10:06:36.515608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:32:56.731620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.337356+0000","last_change":"2024-03-16T10:06:04.227189+0000","last_active":"2024-03-16T10:06:09.337356+0000","last_peered":"2024-03-16T10:06:09.337356+0000","last_clean":"2024-03-16T10:06:09.337356+0000","last_became_active":"2024-03-16T10:06:04.226723+0000","last_became_peered":"2024-03-16T10:06:04.226723+0000","last_unstale":"2024-03-16T10:06:09.337356+0000","last_undegraded":"2024-03-16T10:06:09.337356+0000","last_fullsized":"2024-03-16T10:06:09.337356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:47.873938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929470+0000","last_change":"2024-03-16T10:06:08.529316+0000","last_active":"2024-03-16T10:06:09.929470+0000","last_peered":"2024-03-16T10:06:09.929470+0000","last_clean":"2024-03-16T10:06:09.929470+0000","last_became_active":"2024-03-16T10:06:08.528903+0000","last_became_peered":"2024-03-16T10:06:08.528903+0000","last_unstale":"2024-03-16T10:06:09.929470+0000","last_undegraded":"2024-03-16T10:06:09.929470+0000","last_fullsized":"2024-03-16T10:06:09.929470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:19:33.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.519510+0000","last_change":"2024-03-16T10:06:03.233819+0000","last_active":"2024-03-16T10:06:36.519510+0000","last_peered":"2024-03-16T10:06:36.519510+0000","last_clean":"2024-03-16T10:06:36.519510+0000","last_became_active":"2024-03-16T10:06:03.233593+0000","last_became_peered":"2024-03-16T10:06:03.233593+0000","last_unstale":"2024-03-16T10:06:36.519510+0000","last_undegraded":"2024-03-16T10:06:36.519510+0000","last_fullsized":"2024-03-16T10:06:36.519510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:18.847248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930367+0000","last_change":"2024-03-16T10:06:04.192682+0000","last_active":"2024-03-16T10:06:09.930367+0000","last_peered":"2024-03-16T10:06:09.930367+0000","last_clean":"2024-03-16T10:06:09.930367+0000","last_became_active":"2024-03-16T10:06:04.192305+0000","last_became_peered":"2024-03-16T10:06:04.192305+0000","last_unstale":"2024-03-16T10:06:09.930367+0000","last_undegraded":"2024-03-16T10:06:09.930367+0000","last_fullsized":"2024-03-16T10:06:09.930367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:46.718663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931685+0000","last_change":"2024-03-16T10:06:08.355342+0000","last_active":"2024-03-16T10:06:09.931685+0000","last_peered":"2024-03-16T10:06:09.931685+0000","last_clean":"2024-03-16T10:06:09.931685+0000","last_became_active":"2024-03-16T10:06:08.351547+0000","last_became_peered":"2024-03-16T10:06:08.351547+0000","last_unstale":"2024-03-16T10:06:09.931685+0000","last_undegraded":"2024-03-16T10:06:09.931685+0000","last_fullsized":"2024-03-16T10:06:09.931685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:01:00.777949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.576986+0000","last_change":"2024-03-16T10:06:03.242248+0000","last_active":"2024-03-16T10:06:39.576986+0000","last_peered":"2024-03-16T10:06:39.576986+0000","last_clean":"2024-03-16T10:06:39.576986+0000","last_became_active":"2024-03-16T10:06:03.241842+0000","last_became_peered":"2024-03-16T10:06:03.241842+0000","last_unstale":"2024-03-16T10:06:39.576986+0000","last_undegraded":"2024-03-16T10:06:39.576986+0000","last_fullsized":"2024-03-16T10:06:39.576986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:03:35.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930103+0000","last_change":"2024-03-16T10:06:04.221790+0000","last_active":"2024-03-16T10:06:09.930103+0000","last_peered":"2024-03-16T10:06:09.930103+0000","last_clean":"2024-03-16T10:06:09.930103+0000","last_became_active":"2024-03-16T10:06:04.221475+0000","last_became_peered":"2024-03-16T10:06:04.221475+0000","last_unstale":"2024-03-16T10:06:09.930103+0000","last_undegraded":"2024-03-16T10:06:09.930103+0000","last_fullsized":"2024-03-16T10:06:09.930103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:05:50.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929373+0000","last_change":"2024-03-16T10:06:08.337906+0000","last_active":"2024-03-16T10:06:09.929373+0000","last_peered":"2024-03-16T10:06:09.929373+0000","last_clean":"2024-03-16T10:06:09.929373+0000","last_became_active":"2024-03-16T10:06:08.337444+0000","last_became_peered":"2024-03-16T10:06:08.337444+0000","last_unstale":"2024-03-16T10:06:09.929373+0000","last_undegraded":"2024-03-16T10:06:09.929373+0000","last_fullsized":"2024-03-16T10:06:09.929373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:09:03.997606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:38.557289+0000","last_change":"2024-03-16T10:06:03.235717+0000","last_active":"2024-03-16T10:06:38.557289+0000","last_peered":"2024-03-16T10:06:38.557289+0000","last_clean":"2024-03-16T10:06:38.557289+0000","last_became_active":"2024-03-16T10:06:03.235331+0000","last_became_peered":"2024-03-16T10:06:03.235331+0000","last_unstale":"2024-03-16T10:06:38.557289+0000","last_undegraded":"2024-03-16T10:06:38.557289+0000","last_fullsized":"2024-03-16T10:06:38.557289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:08.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415734+0000","last_change":"2024-03-16T10:06:04.229219+0000","last_active":"2024-03-16T10:06:09.415734+0000","last_peered":"2024-03-16T10:06:09.415734+0000","last_clean":"2024-03-16T10:06:09.415734+0000","last_became_active":"2024-03-16T10:06:04.226850+0000","last_became_peered":"2024-03-16T10:06:04.226850+0000","last_unstale":"2024-03-16T10:06:09.415734+0000","last_undegraded":"2024-03-16T10:06:09.415734+0000","last_fullsized":"2024-03-16T10:06:09.415734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:22:48.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929510+0000","last_change":"2024-03-16T10:06:08.529201+0000","last_active":"2024-03-16T10:06:09.929510+0000","last_peered":"2024-03-16T10:06:09.929510+0000","last_clean":"2024-03-16T10:06:09.929510+0000","last_became_active":"2024-03-16T10:06:08.528782+0000","last_became_peered":"2024-03-16T10:06:08.528782+0000","last_unstale":"2024-03-16T10:06:09.929510+0000","last_undegraded":"2024-03-16T10:06:09.929510+0000","last_fullsized":"2024-03-16T10:06:09.929510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:35:38.267168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:36.515699+0000","last_change":"2024-03-16T10:06:03.227393+0000","last_active":"2024-03-16T10:06:36.515699+0000","last_peered":"2024-03-16T10:06:36.515699+0000","last_clean":"2024-03-16T10:06:36.515699+0000","last_became_active":"2024-03-16T10:06:03.227085+0000","last_became_peered":"2024-03-16T10:06:03.227085+0000","last_unstale":"2024-03-16T10:06:36.515699+0000","last_undegraded":"2024-03-16T10:06:36.515699+0000","last_fullsized":"2024-03-16T10:06:36.515699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:04.302592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599457+0000","last_change":"2024-03-16T10:06:04.217699+0000","last_active":"2024-03-16T10:06:09.599457+0000","last_peered":"2024-03-16T10:06:09.599457+0000","last_clean":"2024-03-16T10:06:09.599457+0000","last_became_active":"2024-03-16T10:06:04.217378+0000","last_became_peered":"2024-03-16T10:06:04.217378+0000","last_unstale":"2024-03-16T10:06:09.599457+0000","last_undegraded":"2024-03-16T10:06:09.599457+0000","last_fullsized":"2024-03-16T10:06:09.599457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:35:23.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333378+0000","last_change":"2024-03-16T10:06:08.344488+0000","last_active":"2024-03-16T10:06:09.333378+0000","last_peered":"2024-03-16T10:06:09.333378+0000","last_clean":"2024-03-16T10:06:09.333378+0000","last_became_active":"2024-03-16T10:06:08.343964+0000","last_became_peered":"2024-03-16T10:06:08.343964+0000","last_unstale":"2024-03-16T10:06:09.333378+0000","last_undegraded":"2024-03-16T10:06:09.333378+0000","last_fullsized":"2024-03-16T10:06:09.333378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:41:09.144062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:40.602741+0000","last_change":"2024-03-16T10:06:03.231805+0000","last_active":"2024-03-16T10:06:40.602741+0000","last_peered":"2024-03-16T10:06:40.602741+0000","last_clean":"2024-03-16T10:06:40.602741+0000","last_became_active":"2024-03-16T10:06:03.230605+0000","last_became_peered":"2024-03-16T10:06:03.230605+0000","last_unstale":"2024-03-16T10:06:40.602741+0000","last_undegraded":"2024-03-16T10:06:40.602741+0000","last_fullsized":"2024-03-16T10:06:40.602741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:21.545277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599913+0000","last_change":"2024-03-16T10:06:04.195674+0000","last_active":"2024-03-16T10:06:09.599913+0000","last_peered":"2024-03-16T10:06:09.599913+0000","last_clean":"2024-03-16T10:06:09.599913+0000","last_became_active":"2024-03-16T10:06:04.195364+0000","last_became_peered":"2024-03-16T10:06:04.195364+0000","last_unstale":"2024-03-16T10:06:09.599913+0000","last_undegraded":"2024-03-16T10:06:09.599913+0000","last_fullsized":"2024-03-16T10:06:09.599913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:40.101280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336455+0000","last_change":"2024-03-16T10:06:08.342352+0000","last_active":"2024-03-16T10:06:09.336455+0000","last_peered":"2024-03-16T10:06:09.336455+0000","last_clean":"2024-03-16T10:06:09.336455+0000","last_became_active":"2024-03-16T10:06:08.341910+0000","last_became_peered":"2024-03-16T10:06:08.341910+0000","last_unstale":"2024-03-16T10:06:09.336455+0000","last_undegraded":"2024-03-16T10:06:09.336455+0000","last_fullsized":"2024-03-16T10:06:09.336455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:23.446076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333887+0000","last_change":"2024-03-16T10:06:03.236814+0000","last_active":"2024-03-16T10:06:09.333887+0000","last_peered":"2024-03-16T10:06:09.333887+0000","last_clean":"2024-03-16T10:06:09.333887+0000","last_became_active":"2024-03-16T10:06:03.236428+0000","last_became_peered":"2024-03-16T10:06:03.236428+0000","last_unstale":"2024-03-16T10:06:09.333887+0000","last_undegraded":"2024-03-16T10:06:09.333887+0000","last_fullsized":"2024-03-16T10:06:09.333887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:44.565924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599690+0000","last_change":"2024-03-16T10:06:04.224725+0000","last_active":"2024-03-16T10:06:09.599690+0000","last_peered":"2024-03-16T10:06:09.599690+0000","last_clean":"2024-03-16T10:06:09.599690+0000","last_became_active":"2024-03-16T10:06:04.224308+0000","last_became_peered":"2024-03-16T10:06:04.224308+0000","last_unstale":"2024-03-16T10:06:09.599690+0000","last_undegraded":"2024-03-16T10:06:09.599690+0000","last_fullsized":"2024-03-16T10:06:09.599690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:55.128000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.332993+0000","last_change":"2024-03-16T10:06:08.330089+0000","last_active":"2024-03-16T10:06:09.332993+0000","last_peered":"2024-03-16T10:06:09.332993+0000","last_clean":"2024-03-16T10:06:09.332993+0000","last_became_active":"2024-03-16T10:06:08.329692+0000","last_became_peered":"2024-03-16T10:06:08.329692+0000","last_unstale":"2024-03-16T10:06:09.332993+0000","last_undegraded":"2024-03-16T10:06:09.332993+0000","last_fullsized":"2024-03-16T10:06:09.332993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:22.141351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:39.577706+0000","last_change":"2024-03-16T10:06:03.247525+0000","last_active":"2024-03-16T10:06:39.577706+0000","last_peered":"2024-03-16T10:06:39.577706+0000","last_clean":"2024-03-16T10:06:39.577706+0000","last_became_active":"2024-03-16T10:06:03.246854+0000","last_became_peered":"2024-03-16T10:06:03.246854+0000","last_unstale":"2024-03-16T10:06:39.577706+0000","last_undegraded":"2024-03-16T10:06:39.577706+0000","last_fullsized":"2024-03-16T10:06:39.577706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:20.648750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.599883+0000","last_change":"2024-03-16T10:06:04.194982+0000","last_active":"2024-03-16T10:06:09.599883+0000","last_peered":"2024-03-16T10:06:09.599883+0000","last_clean":"2024-03-16T10:06:09.599883+0000","last_became_active":"2024-03-16T10:06:04.194696+0000","last_became_peered":"2024-03-16T10:06:04.194696+0000","last_unstale":"2024-03-16T10:06:09.599883+0000","last_undegraded":"2024-03-16T10:06:09.599883+0000","last_fullsized":"2024-03-16T10:06:09.599883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:20.085100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930237+0000","last_change":"2024-03-16T10:06:04.196047+0000","last_active":"2024-03-16T10:06:09.930237+0000","last_peered":"2024-03-16T10:06:09.930237+0000","last_clean":"2024-03-16T10:06:09.930237+0000","last_became_active":"2024-03-16T10:06:04.195742+0000","last_became_peered":"2024-03-16T10:06:04.195742+0000","last_unstale":"2024-03-16T10:06:09.930237+0000","last_undegraded":"2024-03-16T10:06:09.930237+0000","last_fullsized":"2024-03-16T10:06:09.930237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:00:54.239827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930677+0000","last_change":"2024-03-16T10:06:04.193481+0000","last_active":"2024-03-16T10:06:09.930677+0000","last_peered":"2024-03-16T10:06:09.930677+0000","last_clean":"2024-03-16T10:06:09.930677+0000","last_became_active":"2024-03-16T10:06:04.193145+0000","last_became_peered":"2024-03-16T10:06:04.193145+0000","last_unstale":"2024-03-16T10:06:09.930677+0000","last_undegraded":"2024-03-16T10:06:09.930677+0000","last_fullsized":"2024-03-16T10:06:09.930677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:02:10.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.929783+0000","last_change":"2024-03-16T10:06:04.223322+0000","last_active":"2024-03-16T10:06:09.929783+0000","last_peered":"2024-03-16T10:06:09.929783+0000","last_clean":"2024-03-16T10:06:09.929783+0000","last_became_active":"2024-03-16T10:06:04.223116+0000","last_became_peered":"2024-03-16T10:06:04.223116+0000","last_unstale":"2024-03-16T10:06:09.929783+0000","last_undegraded":"2024-03-16T10:06:09.929783+0000","last_fullsized":"2024-03-16T10:06:09.929783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:46.641716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.336831+0000","last_change":"2024-03-16T10:06:04.221926+0000","last_active":"2024-03-16T10:06:09.336831+0000","last_peered":"2024-03-16T10:06:09.336831+0000","last_clean":"2024-03-16T10:06:09.336831+0000","last_became_active":"2024-03-16T10:06:04.221427+0000","last_became_peered":"2024-03-16T10:06:04.221427+0000","last_unstale":"2024-03-16T10:06:09.336831+0000","last_undegraded":"2024-03-16T10:06:09.336831+0000","last_fullsized":"2024-03-16T10:06:09.336831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:51.230757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932117+0000","last_change":"2024-03-16T10:06:04.225518+0000","last_active":"2024-03-16T10:06:09.932117+0000","last_peered":"2024-03-16T10:06:09.932117+0000","last_clean":"2024-03-16T10:06:09.932117+0000","last_became_active":"2024-03-16T10:06:04.225102+0000","last_became_peered":"2024-03-16T10:06:04.225102+0000","last_unstale":"2024-03-16T10:06:09.932117+0000","last_undegraded":"2024-03-16T10:06:09.932117+0000","last_fullsized":"2024-03-16T10:06:09.932117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:38:54.493238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930756+0000","last_change":"2024-03-16T10:06:04.192978+0000","last_active":"2024-03-16T10:06:09.930756+0000","last_peered":"2024-03-16T10:06:09.930756+0000","last_clean":"2024-03-16T10:06:09.930756+0000","last_became_active":"2024-03-16T10:06:04.192641+0000","last_became_peered":"2024-03-16T10:06:04.192641+0000","last_unstale":"2024-03-16T10:06:09.930756+0000","last_undegraded":"2024-03-16T10:06:09.930756+0000","last_fullsized":"2024-03-16T10:06:09.930756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:58.334266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.931573+0000","last_change":"2024-03-16T10:06:04.190020+0000","last_active":"2024-03-16T10:06:09.931573+0000","last_peered":"2024-03-16T10:06:09.931573+0000","last_clean":"2024-03-16T10:06:09.931573+0000","last_became_active":"2024-03-16T10:06:04.189703+0000","last_became_peered":"2024-03-16T10:06:04.189703+0000","last_unstale":"2024-03-16T10:06:09.931573+0000","last_undegraded":"2024-03-16T10:06:09.931573+0000","last_fullsized":"2024-03-16T10:06:09.931573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:23.689558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.416311+0000","last_change":"2024-03-16T10:06:04.234714+0000","last_active":"2024-03-16T10:06:09.416311+0000","last_peered":"2024-03-16T10:06:09.416311+0000","last_clean":"2024-03-16T10:06:09.416311+0000","last_became_active":"2024-03-16T10:06:04.233246+0000","last_became_peered":"2024-03-16T10:06:04.233246+0000","last_unstale":"2024-03-16T10:06:09.416311+0000","last_undegraded":"2024-03-16T10:06:09.416311+0000","last_fullsized":"2024-03-16T10:06:09.416311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:13:31.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333336+0000","last_change":"2024-03-16T10:06:04.219242+0000","last_active":"2024-03-16T10:06:09.333336+0000","last_peered":"2024-03-16T10:06:09.333336+0000","last_clean":"2024-03-16T10:06:09.333336+0000","last_became_active":"2024-03-16T10:06:04.218955+0000","last_became_peered":"2024-03-16T10:06:04.218955+0000","last_unstale":"2024-03-16T10:06:09.333336+0000","last_undegraded":"2024-03-16T10:06:09.333336+0000","last_fullsized":"2024-03-16T10:06:09.333336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:40.760143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.415450+0000","last_change":"2024-03-16T10:06:04.231516+0000","last_active":"2024-03-16T10:06:09.415450+0000","last_peered":"2024-03-16T10:06:09.415450+0000","last_clean":"2024-03-16T10:06:09.415450+0000","last_became_active":"2024-03-16T10:06:04.230328+0000","last_became_peered":"2024-03-16T10:06:04.230328+0000","last_unstale":"2024-03-16T10:06:09.415450+0000","last_undegraded":"2024-03-16T10:06:09.415450+0000","last_fullsized":"2024-03-16T10:06:09.415450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:48:55.678921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.333288+0000","last_change":"2024-03-16T10:06:04.219222+0000","last_active":"2024-03-16T10:06:09.333288+0000","last_peered":"2024-03-16T10:06:09.333288+0000","last_clean":"2024-03-16T10:06:09.333288+0000","last_became_active":"2024-03-16T10:06:04.218903+0000","last_became_peered":"2024-03-16T10:06:04.218903+0000","last_unstale":"2024-03-16T10:06:09.333288+0000","last_undegraded":"2024-03-16T10:06:09.333288+0000","last_fullsized":"2024-03-16T10:06:09.333288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:23.210739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.932090+0000","last_change":"2024-03-16T10:06:04.226025+0000","last_active":"2024-03-16T10:06:09.932090+0000","last_peered":"2024-03-16T10:06:09.932090+0000","last_clean":"2024-03-16T10:06:09.932090+0000","last_became_active":"2024-03-16T10:06:04.225657+0000","last_became_peered":"2024-03-16T10:06:04.225657+0000","last_unstale":"2024-03-16T10:06:09.932090+0000","last_undegraded":"2024-03-16T10:06:09.932090+0000","last_fullsized":"2024-03-16T10:06:09.932090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:08:15.163902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-16T10:06:09.930517+0000","last_change":"2024-03-16T10:06:04.198047+0000","last_active":"2024-03-16T10:06:09.930517+0000","last_peered":"2024-03-16T10:06:09.930517+0000","last_clean":"2024-03-16T10:06:09.930517+0000","last_became_active":"2024-03-16T10:06:04.197751+0000","last_became_peered":"2024-03-16T10:06:04.197751+0000","last_unstale":"2024-03-16T10:06:09.930517+0000","last_undegraded":"2024-03-16T10:06:09.930517+0000","last_fullsized":"2024-03-16T10:06:09.930517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:52.703845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":215634,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48013,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":217,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":63,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":85498,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350997,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1095902,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":485484,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":485862,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1082359,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":480988,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480960,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":505770,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480696,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480932,"data_compressed":14690,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1070816,"data_compressed":51175,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":44033,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:06:43.743 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T10:06:43.743 INFO:tasks.cephadm:Setting up client nodes... 2024-03-16T10:06:43.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T10:06:43.860 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:43 smithi039 ceph-mon[31510]: pgmap v335: 129 pgs: 129 active+clean; 624 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T10:06:43.860 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:43 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1148885575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:43 smithi105 ceph-mon[36559]: pgmap v335: 129 pgs: 129 active+clean; 624 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T10:06:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:43 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1148885575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:43.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:43 smithi194 ceph-mon[36445]: pgmap v335: 129 pgs: 129 active+clean; 624 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-03-16T10:06:43.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:43 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1148885575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:06:44.612 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:06:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:06:44.288+0000 7f8ac9e70700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-16T10:06:44.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:44 smithi039 ceph-mon[31510]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:44.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:44 smithi039 ceph-mon[31510]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:44.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:44 smithi105 ceph-mon[36559]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:44.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:44 smithi105 ceph-mon[36559]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:44 smithi194 ceph-mon[36445]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:44 smithi194 ceph-mon[36445]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:06:45.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:45 smithi039 ceph-mon[31510]: pgmap v336: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:45.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:45 smithi194 ceph-mon[36445]: pgmap v336: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:46.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:45 smithi105 ceph-mon[36559]: pgmap v336: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:46.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 -- 172.21.15.39:0/1242439838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 msgr2=0x7fc474102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 --2- 172.21.15.39:0/1242439838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc474102590 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc4640099c0 tx=0x7fc464009cd0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 -- 172.21.15.39:0/1242439838 shutdown_connections 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 --2- 172.21.15.39:0/1242439838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 0x7fc47410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 --2- 172.21.15.39:0/1242439838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc474102590 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 --2- 172.21.15.39:0/1242439838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 0x7fc4741037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 -- 172.21.15.39:0/1242439838 >> 172.21.15.39:0/1242439838 conn(0x7fc4740fd710 msgr2=0x7fc4740ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 -- 172.21.15.39:0/1242439838 shutdown_connections 2024-03-16T10:06:46.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.599+0000 7fc479600700 1 -- 172.21.15.39:0/1242439838 wait complete. 2024-03-16T10:06:46.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 Processor -- start 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 -- start start 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 0x7fc474198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 0x7fc47419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410dda0 con 0x7fc474103d10 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410dc20 con 0x7fc474102170 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.601+0000 7fc479600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc47410df20 con 0x7fc474103370 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:40882/0 (socket says 172.21.15.39:40882) 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 -- 172.21.15.39:0/3888443565 learned_addr learned my addr 172.21.15.39:0/3888443565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 0x7fc474198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:46.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc478c00700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 0x7fc47419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 -- 172.21.15.39:0/3888443565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 msgr2=0x7fc474198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 0x7fc474198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 -- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 msgr2=0x7fc47419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 0x7fc47419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.602+0000 7fc473e00700 1 -- 172.21.15.39:0/3888443565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47419ff50 con 0x7fc474102170 2024-03-16T10:06:46.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc473e00700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc46400b580 tx=0x7fc46400bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464003e30 con 0x7fc474102170 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46400f460 con 0x7fc474102170 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc464017650 con 0x7fc474102170 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741a01e0 con 0x7fc474102170 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.603+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741a0690 con 0x7fc474102170 2024-03-16T10:06:46.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.605+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc474063900 con 0x7fc474102170 2024-03-16T10:06:46.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.607+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc4640052d0 con 0x7fc474102170 2024-03-16T10:06:46.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.608+0000 7fc470c00700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 0x7fc4580a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:46.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.608+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc4640cfce0 con 0x7fc474102170 2024-03-16T10:06:46.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.608+0000 7fc473400700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 0x7fc4580a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:46.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.609+0000 7fc473400700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 0x7fc4580a4f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc460009680 tx=0x7fc460009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:46.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.613+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc464027060 con 0x7fc474102170 2024-03-16T10:06:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.964+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc47402db70 con 0x7fc474102170 2024-03-16T10:06:46.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.971+0000 7fc470c00700 1 -- 172.21.15.39:0/3888443565 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc464013070 con 0x7fc474102170 2024-03-16T10:06:46.973 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-03-16T10:06:46.973 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA2b/VlAqmeORAAkev7Yp6n59R2ApWV47TDKw== 2024-03-16T10:06:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 msgr2=0x7fc4580a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 0x7fc4580a4f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc460009680 tx=0x7fc460009260 comp rx=0 tx=0).stop 2024-03-16T10:06:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 msgr2=0x7fc4741982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc46400b580 tx=0x7fc46400bee0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 shutdown_connections 2024-03-16T10:06:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc4580a2a50 0x7fc4580a4f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc474103d10 0x7fc47419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc474102170 0x7fc4741982f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 --2- 172.21.15.39:0/3888443565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc474103370 0x7fc474198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.974+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 >> 172.21.15.39:0/3888443565 conn(0x7fc4740fd710 msgr2=0x7fc4741065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.975+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 shutdown_connections 2024-03-16T10:06:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:46.975+0000 7fc479600700 1 -- 172.21.15.39:0/3888443565 wait complete. 2024-03-16T10:06:47.733 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:47 smithi039 ceph-mon[31510]: pgmap v337: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:47.733 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:47 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3888443565' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:47.733 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:47 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:47.733 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:47 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:06:47.809 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:06:47.809 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-16T10:06:47.809 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T10:06:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:47 smithi194 ceph-mon[36445]: pgmap v337: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:47 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3888443565' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:47 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:47.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:47 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:06:47.959 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T10:06:47.959 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-16T10:06:47.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-16T10:06:48.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:47 smithi105 ceph-mon[36559]: pgmap v337: 129 pgs: 129 active+clean; 625 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2024-03-16T10:06:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:47 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3888443565' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:47 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:06:48.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:47 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:06:48.477 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:06:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:49 smithi039 ceph-mon[31510]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-16T10:06:49.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:49 smithi194 ceph-mon[36445]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-16T10:06:50.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:49 smithi105 ceph-mon[36559]: pgmap v338: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-16T10:06:50.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 -- 172.21.15.39:0/2041141882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac06a920 msgr2=0x7f34ac10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:50.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2041141882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac06a920 0x7f34ac10f160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009fc0 tx=0x7f34a400b330 comp rx=0 tx=0).stop 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 -- 172.21.15.39:0/2041141882 shutdown_connections 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2041141882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac06a920 0x7f34ac10f160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2041141882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac108320 0x7f34ac108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2041141882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac069f80 0x7f34ac06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.673+0000 7f34b0a00700 1 -- 172.21.15.39:0/2041141882 >> 172.21.15.39:0/2041141882 conn(0x7f34ac078100 msgr2=0x7f34ac078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.674+0000 7f34b0a00700 1 -- 172.21.15.39:0/2041141882 shutdown_connections 2024-03-16T10:06:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.674+0000 7f34b0a00700 1 -- 172.21.15.39:0/2041141882 wait complete. 2024-03-16T10:06:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.675+0000 7f34b0a00700 1 Processor -- start 2024-03-16T10:06:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 -- start start 2024-03-16T10:06:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac069f80 0x7f34ac1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac108320 0x7f34ac19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1125f0 con 0x7f34ac069f80 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112470 con 0x7f34ac108320 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.676+0000 7f34b0a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112770 con 0x7f34ac06a920 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:38222/0 (socket says 172.21.15.39:38222) 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 -- 172.21.15.39:0/2276991215 learned_addr learned my addr 172.21.15.39:0/2276991215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34aa800700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac108320 0x7f34ac19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9e00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac069f80 0x7f34ac1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 -- 172.21.15.39:0/2276991215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac108320 msgr2=0x7f34ac19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac108320 0x7f34ac19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 -- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac069f80 msgr2=0x7f34ac1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac069f80 0x7f34ac1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.677+0000 7f34a9400700 1 -- 172.21.15.39:0/2276991215 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac19b1a0 con 0x7f34ac06a920 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34a9400700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f349800ec10 tx=0x7f349800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800cc90 con 0x7f34ac06a920 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349800ce30 con 0x7f34ac06a920 2024-03-16T10:06:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac19b490 con 0x7f34ac06a920 2024-03-16T10:06:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498010690 con 0x7f34ac06a920 2024-03-16T10:06:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.678+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a7750 con 0x7f34ac06a920 2024-03-16T10:06:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.679+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac068f90 con 0x7f34ac06a920 2024-03-16T10:06:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.681+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f3498003680 con 0x7f34ac06a920 2024-03-16T10:06:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.682+0000 7f34a2a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 0x7f34900a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.683+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f34980cf1b0 con 0x7f34ac06a920 2024-03-16T10:06:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.683+0000 7f34a9e00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 0x7f34900a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.684+0000 7f34a9e00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 0x7f34900a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34ac19bb80 tx=0x7f349c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.691+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f349809be50 con 0x7f34ac06a920 2024-03-16T10:06:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.978+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f34ac19bf70 con 0x7f34ac06a920 2024-03-16T10:06:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.983+0000 7f34a2a00700 1 -- 172.21.15.39:0/2276991215 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262660 (secure 0 0 0) 0x7f34ac19bf70 con 0x7f34ac06a920 2024-03-16T10:06:50.988 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.988+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 msgr2=0x7f34900a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.988+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 0x7f34900a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f34ac19bb80 tx=0x7f349c009260 comp rx=0 tx=0).stop 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.988+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 msgr2=0x7f34ac1a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.988+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f349800ec10 tx=0x7f349800ef20 comp rx=0 tx=0).stop 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 shutdown_connections 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f34900a29a0 0x7f34900a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34ac108320 0x7f34ac19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34ac069f80 0x7f34ac1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 --2- 172.21.15.39:0/2276991215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34ac06a920 0x7f34ac1a0dc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 >> 172.21.15.39:0/2276991215 conn(0x7f34ac078100 msgr2=0x7f34ac10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 shutdown_connections 2024-03-16T10:06:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:50.989+0000 7f34b0a00700 1 -- 172.21.15.39:0/2276991215 wait complete. 2024-03-16T10:06:51.874 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":1082942412},{"type":"v1","addr":"172.21.15.39:6801","nonce":1082942412}]},"active_addr":"172.21.15.39:6801/1082942412","active_change":"2024-03-16T09:58:12.056692+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2375807750}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":332905917}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3265323877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2792410890}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4100918223}]}]} 2024-03-16T10:06:51.879 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-16T10:06:51.879 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-16T10:06:51.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:06:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:51 smithi194 ceph-mon[36445]: pgmap v339: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-03-16T10:06:51.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2276991215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T10:06:51.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:51 smithi039 ceph-mon[31510]: pgmap v339: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-03-16T10:06:51.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2276991215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T10:06:52.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:51 smithi105 ceph-mon[36559]: pgmap v339: 129 pgs: 129 active+clean; 629 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-03-16T10:06:52.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2276991215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T10:06:52.352 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:06:53.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:53 smithi194 ceph-mon[36445]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-16T10:06:54.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:53 smithi039 ceph-mon[31510]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-16T10:06:54.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:53 smithi105 ceph-mon[36559]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 -- 172.21.15.39:0/1504613361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e50100f80 msgr2=0x7f9e50101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/1504613361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e50100f80 0x7f9e50101360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009990 tx=0x7f9e40009ca0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 -- 172.21.15.39:0/1504613361 shutdown_connections 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/1504613361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e5010eb30 0x7f9e50110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/1504613361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e50100f80 0x7f9e50101360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/1504613361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50101930 0x7f9e5010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 -- 172.21.15.39:0/1504613361 >> 172.21.15.39:0/1504613361 conn(0x7f9e500fcba0 msgr2=0x7f9e500fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 -- 172.21.15.39:0/1504613361 shutdown_connections 2024-03-16T10:06:54.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.544+0000 7f9e4fe00700 1 -- 172.21.15.39:0/1504613361 wait complete. 2024-03-16T10:06:54.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.545+0000 7f9e4fe00700 1 Processor -- start 2024-03-16T10:06:54.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.545+0000 7f9e4fe00700 1 -- start start 2024-03-16T10:06:54.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50100f80 0x7f9e501a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:54.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:54.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e5010eb30 0x7f9e5019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:54.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e50114750 con 0x7f9e50100f80 2024-03-16T10:06:54.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e501145d0 con 0x7f9e5010eb30 2024-03-16T10:06:54.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4fe00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9e501148d0 con 0x7f9e50101930 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e47e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4ea00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50100f80 0x7f9e501a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e47e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:38240/0 (socket says 172.21.15.39:38240) 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e47e00700 1 -- 172.21.15.39:0/247190747 learned_addr learned my addr 172.21.15.39:0/247190747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.546+0000 7f9e4f400700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e5010eb30 0x7f9e5019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 -- 172.21.15.39:0/247190747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e5010eb30 msgr2=0x7f9e5019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e5010eb30 0x7f9e5019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 -- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50100f80 msgr2=0x7f9e501a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50100f80 0x7f9e501a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 -- 172.21.15.39:0/247190747 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5019b190 con 0x7f9e50101930 2024-03-16T10:06:54.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47e00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9a0 tx=0x7f9e3800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800ca10 con 0x7f9e50101930 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5019b480 con 0x7f9e50101930 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3800cbb0 con 0x7f9e50101930 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.547+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38018680 con 0x7f9e50101930 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.548+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e501a76f0 con 0x7f9e50101930 2024-03-16T10:06:54.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.549+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e50068f90 con 0x7f9e50101930 2024-03-16T10:06:54.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.551+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f9e38047a90 con 0x7f9e50101930 2024-03-16T10:06:54.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.552+0000 7f9e47400700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 0x7f9e300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:54.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.552+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9e380cf0c0 con 0x7f9e50101930 2024-03-16T10:06:54.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.552+0000 7f9e4ea00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 0x7f9e300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:54.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.553+0000 7f9e4ea00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 0x7f9e300a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e5010f2d0 tx=0x7f9e4000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.558+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e3809bd60 con 0x7f9e50101930 2024-03-16T10:06:54.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.813+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e5002db70 con 0x7f9e50101930 2024-03-16T10:06:54.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.815+0000 7f9e47400700 1 -- 172.21.15.39:0/247190747 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7f9e3809b660 con 0x7f9e50101930 2024-03-16T10:06:54.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:54.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:54.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 msgr2=0x7f9e300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:54.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 0x7f9e300a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e5010f2d0 tx=0x7f9e4000b260 comp rx=0 tx=0).stop 2024-03-16T10:06:54.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 msgr2=0x7f9e501a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:54.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9a0 tx=0x7f9e3800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 shutdown_connections 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9e300a2920 0x7f9e300a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e5010eb30 0x7f9e5019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e50100f80 0x7f9e501a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 --2- 172.21.15.39:0/247190747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e50101930 0x7f9e501a0d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 >> 172.21.15.39:0/247190747 conn(0x7f9e500fcba0 msgr2=0x7f9e5010fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 shutdown_connections 2024-03-16T10:06:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:54.819+0000 7f9e4fe00700 1 -- 172.21.15.39:0/247190747 wait complete. 2024-03-16T10:06:55.601 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-16T10:06:55.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:06:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:55 smithi194 ceph-mon[36445]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-16T10:06:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/247190747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:55 smithi039 ceph-mon[31510]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-16T10:06:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/247190747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:06:56.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:55 smithi105 ceph-mon[36559]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-16T10:06:56.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/247190747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:57 smithi039 ceph-mon[31510]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-03-16T10:06:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:57 smithi194 ceph-mon[36445]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-03-16T10:06:58.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:57 smithi105 ceph-mon[36559]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-03-16T10:06:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3036377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4108300 msgr2=0x7ff3c41086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3036377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4108300 0x7ff3c41086e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80099c0 tx=0x7ff3b8009cd0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3036377538 shutdown_connections 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3036377538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 0x7ff3c410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3036377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4108300 0x7ff3c41086e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3036377538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4102320 0x7ff3c41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.265+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3036377538 >> 172.21.15.39:0/3036377538 conn(0x7ff3c4079400 msgr2=0x7ff3c4079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.266+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3036377538 shutdown_connections 2024-03-16T10:06:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.266+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3036377538 wait complete. 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 Processor -- start 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 -- start start 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 0x7ff3c41a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4108300 0x7ff3c419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41125d0 con 0x7ff3c4108300 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4112450 con 0x7ff3c4102320 2024-03-16T10:06:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.267+0000 7ff3cbc00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4112750 con 0x7ff3c4102ce0 2024-03-16T10:06:58.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39576/0 (socket says 172.21.15.39:39576) 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 -- 172.21.15.39:0/3424362836 learned_addr learned my addr 172.21.15.39:0/3424362836 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c3e00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 0x7ff3c41a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9c00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4108300 0x7ff3c419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 -- 172.21.15.39:0/3424362836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 msgr2=0x7ff3c41a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 0x7ff3c41a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 -- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4108300 msgr2=0x7ff3c419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4108300 0x7ff3c419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 -- 172.21.15.39:0/3424362836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c419b130 con 0x7ff3c4102320 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.268+0000 7ff3c9200700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005910 tx=0x7ff3b800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.269+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8027070 con 0x7ff3c4102320 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.269+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8003680 con 0x7ff3c4102320 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.269+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c419b3c0 con 0x7ff3c4102320 2024-03-16T10:06:58.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.269+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b80173f0 con 0x7ff3c4102320 2024-03-16T10:06:58.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.269+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41a76e0 con 0x7ff3c4102320 2024-03-16T10:06:58.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.270+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c4068f90 con 0x7ff3c4102320 2024-03-16T10:06:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.274+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff3b804f9e0 con 0x7ff3c4102320 2024-03-16T10:06:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.275+0000 7ff3c1600700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 0x7ff3ac0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:06:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.275+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff3b80cfa40 con 0x7ff3c4102320 2024-03-16T10:06:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.275+0000 7ff3c3e00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 0x7ff3ac0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:06:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.276+0000 7ff3c3e00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 0x7ff3ac0a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4079c70 tx=0x7ff3b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:06:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.276+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3b8013070 con 0x7ff3c4102320 2024-03-16T10:06:58.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.503+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3c41a78b0 con 0x7ff3c4102320 2024-03-16T10:06:58.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.505+0000 7ff3c1600700 1 -- 172.21.15.39:0/3424362836 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24117 (secure 0 0 0) 0x7ff3b809c770 con 0x7ff3c4102320 2024-03-16T10:06:58.506 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:06:58.506 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:06:08.798657+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:06:58.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 msgr2=0x7ff3ac0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:58.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 0x7ff3ac0a4e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4079c70 tx=0x7ff3b00073d0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 msgr2=0x7ff3c41a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:06:58.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005910 tx=0x7ff3b800fa00 comp rx=0 tx=0).stop 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 shutdown_connections 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3ac0a29a0 0x7ff3ac0a4e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c4108300 0x7ff3c419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c4102320 0x7ff3c41a0810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 --2- 172.21.15.39:0/3424362836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3c4102ce0 0x7ff3c41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 >> 172.21.15.39:0/3424362836 conn(0x7ff3c4079400 msgr2=0x7ff3c4074730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.509+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 shutdown_connections 2024-03-16T10:06:58.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:06:58.510+0000 7ff3cbc00700 1 -- 172.21.15.39:0/3424362836 wait complete. 2024-03-16T10:06:58.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3424362836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:58.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:58 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3424362836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:59.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:58 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3424362836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:06:59.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-16T10:06:59.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-16T10:06:59.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-16T10:06:59.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-16T10:06:59.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-16T10:06:59.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-16T10:06:59.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-16T10:06:59.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-16T10:06:59.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-16T10:06:59.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-16T10:06:59.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-16T10:06:59.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-16T10:06:59.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:06:59 smithi194 ceph-mon[36445]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-16T10:07:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:06:59 smithi039 ceph-mon[31510]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-16T10:07:00.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:06:59 smithi105 ceph-mon[36559]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-03-16T10:07:00.503 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:00.539 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:00.669 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:01 smithi194 ceph-mon[36445]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T10:07:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:01 smithi039 ceph-mon[31510]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T10:07:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:01 smithi105 ceph-mon[36559]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T10:07:03.863 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:03 smithi039 ceph-mon[31510]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-16T10:07:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:03 smithi194 ceph-mon[36445]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-16T10:07:04.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:03 smithi105 ceph-mon[36559]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-16T10:07:05.075 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:05.075 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:05.076 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:05.077 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:05 smithi039 ceph-mon[31510]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:05 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:07:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:05 smithi194 ceph-mon[36445]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:05 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:07:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:05 smithi105 ceph-mon[36559]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:06.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:05 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:07:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:07 smithi194 ceph-mon[36445]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T10:07:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:07 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:07 smithi039 ceph-mon[31510]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T10:07:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:07 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:07 smithi105 ceph-mon[36559]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T10:07:08.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:08.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:07 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 -- 172.21.15.39:0/1874476658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b3406aeb0 msgr2=0x7f7b3406b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1874476658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b3406aeb0 0x7f7b3406b330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009990 tx=0x7f7b20009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 -- 172.21.15.39:0/1874476658 shutdown_connections 2024-03-16T10:07:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1874476658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34104d00 0x7f7b34105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1874476658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b341071e0 0x7f7b3406a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1874476658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b3406aeb0 0x7f7b3406b330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 -- 172.21.15.39:0/1874476658 >> 172.21.15.39:0/1874476658 conn(0x7f7b34078140 msgr2=0x7f7b34078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 -- 172.21.15.39:0/1874476658 shutdown_connections 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.341+0000 7f7b38c00700 1 -- 172.21.15.39:0/1874476658 wait complete. 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 Processor -- start 2024-03-16T10:07:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 -- start start 2024-03-16T10:07:09.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b3406aeb0 0x7f7b3419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34104d00 0x7f7b3419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 0x7f7b341a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.343+0000 7f7b38c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34118b90 con 0x7f7b3406aeb0 2024-03-16T10:07:09.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b38c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34118a10 con 0x7f7b34104d00 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b38c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34118d10 con 0x7f7b341071e0 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b31e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b3406aeb0 0x7f7b3419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 0x7f7b341a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b31e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b3406aeb0 0x7f7b3419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47204/0 (socket says 172.21.15.39:47204) 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 0x7f7b341a3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:46776/0 (socket says 172.21.15.39:46776) 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b31e00700 1 -- 172.21.15.39:0/1725504328 learned_addr learned my addr 172.21.15.39:0/1725504328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b31400700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34104d00 0x7f7b3419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 -- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34104d00 msgr2=0x7f7b3419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b34104d00 0x7f7b3419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 -- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b3406aeb0 msgr2=0x7f7b3419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b3406aeb0 0x7f7b3419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.344+0000 7f7b32800700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b341a4040 con 0x7f7b341071e0 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b32800700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 0x7f7b341a3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00ef90 tx=0x7f7b2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003a00 con 0x7f7b341071e0 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2c003ba0 con 0x7f7b341071e0 2024-03-16T10:07:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b341a4330 con 0x7f7b341071e0 2024-03-16T10:07:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c004770 con 0x7f7b341071e0 2024-03-16T10:07:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.345+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341abbb0 con 0x7f7b341071e0 2024-03-16T10:07:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.346+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b34104740 con 0x7f7b341071e0 2024-03-16T10:07:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.348+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7b2c012050 con 0x7f7b341071e0 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.349+0000 7f7b2aa00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7b180a29a0 0x7f7b180a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7b2c0ce720 con 0x7f7b341071e0 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b31e00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7b180a29a0 0x7f7b180a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b2aa00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] conn(0x7f7b180ad660 0x7f7b180afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b180b0170 con 0x7f7b180ad660 2024-03-16T10:07:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7b2c0d5050 con 0x7f7b341071e0 2024-03-16T10:07:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.350+0000 7f7b31400700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] conn(0x7f7b180ad660 0x7f7b180afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.351+0000 7f7b31e00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7b180a29a0 0x7f7b180a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ca0 tx=0x7f7b24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.351+0000 7f7b31400700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] conn(0x7f7b180ad660 0x7f7b180afaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.352+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== osd.7 v2:172.21.15.105:6816/2696110582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b180b0170 con 0x7f7b180ad660 2024-03-16T10:07:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.370+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 --> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b3406a220 con 0x7f7b180ad660 2024-03-16T10:07:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.370+0000 7f7b2aa00700 1 -- 172.21.15.39:0/1725504328 <== osd.7 v2:172.21.15.105:6816/2696110582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b3406a220 con 0x7f7b180ad660 2024-03-16T10:07:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.370+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] conn(0x7f7b180ad660 msgr2=0x7f7b180afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.370+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] conn(0x7f7b180ad660 0x7f7b180afaa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7b180a29a0 msgr2=0x7f7b180a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7b180a29a0 0x7f7b180a4e60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ca0 tx=0x7f7b24009260 comp rx=0 tx=0).stop 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 msgr2=0x7f7b341a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 --2- 172.21.15.39:0/1725504328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b341071e0 0x7f7b341a3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00ef90 tx=0x7f7b2c00c3a0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b32800700 1 -- 172.21.15.39:0/1725504328 reap_dead start 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 shutdown_connections 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 >> 172.21.15.39:0/1725504328 conn(0x7f7b34078140 msgr2=0x7f7b34103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 shutdown_connections 2024-03-16T10:07:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:09.371+0000 7f7b38c00700 1 -- 172.21.15.39:0/1725504328 wait complete. 2024-03-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.852+0000 7f1942a00700 1 -- 172.21.15.39:0/645824207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074500 msgr2=0x7f193c0748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.852+0000 7f1942a00700 1 --2- 172.21.15.39:0/645824207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074500 0x7f193c0748e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f192c009990 tx=0x7f192c009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 -- 172.21.15.39:0/645824207 shutdown_connections 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 --2- 172.21.15.39:0/645824207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 0x7f193c115890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 --2- 172.21.15.39:0/645824207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074500 0x7f193c0748e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 --2- 172.21.15.39:0/645824207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f193c074e20 0x7f193c112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 -- 172.21.15.39:0/645824207 >> 172.21.15.39:0/645824207 conn(0x7f193c079430 msgr2=0x7f193c079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 -- 172.21.15.39:0/645824207 shutdown_connections 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.853+0000 7f1942a00700 1 -- 172.21.15.39:0/645824207 wait complete. 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 Processor -- start 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 -- start start 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f193c074500 0x7f193c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 0x7f193c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 0x7f193c1a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118ce0 con 0x7f193c074500 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118b60 con 0x7f193c074e20 2024-03-16T10:07:09.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1942a00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f193c118e60 con 0x7f193c1134a0 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f193b400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 0x7f193c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f193be00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f193c074500 0x7f193c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.854+0000 7f1940a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 0x7f193c1a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 0x7f193c19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:45516/0 (socket says 172.21.15.39:45516) 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 -- 172.21.15.39:0/1074549688 learned_addr learned my addr 172.21.15.39:0/1074549688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f1940a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 0x7f193c1a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:46756/0 (socket says 172.21.15.39:46756) 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 -- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 msgr2=0x7f193c1a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f193c1134a0 0x7f193c1a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 -- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f193c074500 msgr2=0x7f193c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f193c074500 0x7f193c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a4100 con 0x7f193c074e20 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f193b400700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 0x7f193c19f630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f192800e970 tx=0x7f192800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.855+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928004220 con 0x7f193c074e20 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.856+0000 7f1942a00700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a43f0 con 0x7f193c074e20 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.856+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19280043c0 con 0x7f193c074e20 2024-03-16T10:07:09.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.856+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928003dc0 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.856+0000 7f1942a00700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1abc70 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.857+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f193c0629a0 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.859+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f192800c7e0 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.860+0000 7f1938c00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f19200a2920 0x7f19200a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.860+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f19280cf1f0 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.860+0000 7f193be00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f19200a2920 0x7f19200a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.861+0000 7f1938c00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] conn(0x7f19200ad5e0 0x7f19200afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.861+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19200b00f0 con 0x7f19200ad5e0 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.861+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f19280cf5a0 con 0x7f193c074e20 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.861+0000 7f1940a00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] conn(0x7f19200ad5e0 0x7f19200afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.861+0000 7f193be00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f19200a2920 0x7f19200a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f192c006010 tx=0x7f192c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.862+0000 7f1940a00700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] conn(0x7f19200ad5e0 0x7f19200afa20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.863+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== osd.5 v2:172.21.15.194:6808/846499342 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19200b00f0 con 0x7f19200ad5e0 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.881+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 --> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f193c062da0 con 0x7f19200ad5e0 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.882+0000 7f1938c00700 1 -- 172.21.15.39:0/1074549688 <== osd.5 v2:172.21.15.194:6808/846499342 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f193c062da0 con 0x7f19200ad5e0 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.882+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] conn(0x7f19200ad5e0 msgr2=0x7f19200afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.882+0000 7f1932000700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] conn(0x7f19200ad5e0 0x7f19200afa20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f19200a2920 msgr2=0x7f19200a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f19200a2920 0x7f19200a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f192c006010 tx=0x7f192c00b260 comp rx=0 tx=0).stop 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 msgr2=0x7f193c19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 --2- 172.21.15.39:0/1074549688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f193c074e20 0x7f193c19f630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f192800e970 tx=0x7f192800ec80 comp rx=0 tx=0).stop 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1940a00700 1 -- 172.21.15.39:0/1074549688 reap_dead start 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 shutdown_connections 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 >> 172.21.15.39:0/1074549688 conn(0x7f193c079430 msgr2=0x7f193c112260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 shutdown_connections 2024-03-16T10:07:09.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.883+0000 7f1932000700 1 -- 172.21.15.39:0/1074549688 wait complete. 2024-03-16T10:07:09.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.710+0000 7f0098600700 1 -- 172.21.15.39:0/3766292361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f009006b420 msgr2=0x7f0090106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.710+0000 7f0098600700 1 --2- 172.21.15.39:0/3766292361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f009006b420 0x7f0090106f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f007c009990 tx=0x7f007c009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 -- 172.21.15.39:0/3766292361 shutdown_connections 2024-03-16T10:07:09.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 --2- 172.21.15.39:0/3766292361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0090107650 0x7f009010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 --2- 172.21.15.39:0/3766292361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 0x7f009006ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 --2- 172.21.15.39:0/3766292361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f009006b420 0x7f0090106f20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 -- 172.21.15.39:0/3766292361 >> 172.21.15.39:0/3766292361 conn(0x7f0090077220 msgr2=0x7f0090077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 -- 172.21.15.39:0/3766292361 shutdown_connections 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.711+0000 7f0098600700 1 -- 172.21.15.39:0/3766292361 wait complete. 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 Processor -- start 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 -- start start 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 0x7f00901a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 0x7f00901a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0090107650 0x7f009019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f00901125f0 con 0x7f009006aa70 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0090112470 con 0x7f0090107650 2024-03-16T10:07:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.712+0000 7f0098600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0090112770 con 0x7f009006b420 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 0x7f00901a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 0x7f00901a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:46746/0 (socket says 172.21.15.39:46746) 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 -- 172.21.15.39:0/1818561158 learned_addr learned my addr 172.21.15.39:0/1818561158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095c00700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 0x7f00901a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0096600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0090107650 0x7f009019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 -- 172.21.15.39:0/1818561158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0090107650 msgr2=0x7f009019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0090107650 0x7f009019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 -- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 msgr2=0x7f00901a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 0x7f00901a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095200700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009019b450 con 0x7f009006b420 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0096600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0090107650 0x7f009019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.713+0000 7f0095c00700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f009006aa70 0x7f00901a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:09.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f0095200700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 0x7f00901a0f90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f007c0098e0 tx=0x7f007c005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c027070 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007c022470 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c017650 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009019b6e0 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.714+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a7800 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.715+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f009010dfb0 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.718+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f007c02a070 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f008e000700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f00780a2920 0x7f00780a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f007c0d0160 con 0x7f009006b420 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f0095c00700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f00780a2920 0x7f00780a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f008e000700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] conn(0x7f00780ad5e0 0x7f00780afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f007c015ec0 con 0x7f00780ad5e0 2024-03-16T10:07:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.719+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f007c0d0510 con 0x7f009006b420 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.720+0000 7f0096600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] conn(0x7f00780ad5e0 0x7f00780afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.720+0000 7f0095c00700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f00780a2920 0x7f00780a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0084009cb0 tx=0x7f0084009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.721+0000 7f0096600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] conn(0x7f00780ad5e0 0x7f00780afa20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.722+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== osd.0 v2:172.21.15.39:6802/1765190956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f007c015ec0 con 0x7f00780ad5e0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.741+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 --> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0090103a00 con 0x7f00780ad5e0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f008e000700 1 -- 172.21.15.39:0/1818561158 <== osd.0 v2:172.21.15.39:6802/1765190956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0090103a00 con 0x7f00780ad5e0 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] conn(0x7f00780ad5e0 msgr2=0x7f00780afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] conn(0x7f00780ad5e0 0x7f00780afa20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f00780a2920 msgr2=0x7f00780a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f00780a2920 0x7f00780a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0084009cb0 tx=0x7f0084009260 comp rx=0 tx=0).stop 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 msgr2=0x7f00901a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.742+0000 7f0098600700 1 --2- 172.21.15.39:0/1818561158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f009006b420 0x7f00901a0f90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f007c0098e0 tx=0x7f007c005640 comp rx=0 tx=0).stop 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.743+0000 7f0096600700 1 -- 172.21.15.39:0/1818561158 reap_dead start 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.743+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 shutdown_connections 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.743+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 >> 172.21.15.39:0/1818561158 conn(0x7f0090077220 msgr2=0x7f009010d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.743+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 shutdown_connections 2024-03-16T10:07:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:08.743+0000 7f0098600700 1 -- 172.21.15.39:0/1818561158 wait complete. 2024-03-16T10:07:10.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:09 smithi194 ceph-mon[36445]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-16T10:07:10.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:09 smithi039 ceph-mon[31510]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-16T10:07:10.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:09 smithi105 ceph-mon[36559]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-16T10:07:11.379 INFO:teuthology.orchestra.run.smithi039.stdout:167503724604 2024-03-16T10:07:11.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-16T10:07:11.713 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:12.003 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:12.005 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:12.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:11 smithi039 ceph-mon[31510]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:12.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:11 smithi194 ceph-mon[36445]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:12.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:11 smithi105 ceph-mon[36559]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.495+0000 7f4e9c200700 1 -- 172.21.15.39:0/1801527324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 msgr2=0x7f4e94106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.495+0000 7f4e9c200700 1 --2- 172.21.15.39:0/1801527324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e94106f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009990 tx=0x7f4e80009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 -- 172.21.15.39:0/1801527324 shutdown_connections 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 --2- 172.21.15.39:0/1801527324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e94107650 0x7f4e9410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 --2- 172.21.15.39:0/1801527324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e94102580 0x7f4e94102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 --2- 172.21.15.39:0/1801527324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e94106f70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 -- 172.21.15.39:0/1801527324 >> 172.21.15.39:0/1801527324 conn(0x7f4e94079430 msgr2=0x7f4e94079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 -- 172.21.15.39:0/1801527324 shutdown_connections 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.496+0000 7f4e9c200700 1 -- 172.21.15.39:0/1801527324 wait complete. 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 Processor -- start 2024-03-16T10:07:12.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 -- start start 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e94102580 0x7f4e9419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e9419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e94107650 0x7f4e94198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94112660 con 0x7f4e94102580 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941124e0 con 0x7f4e94102f30 2024-03-16T10:07:12.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.497+0000 7f4e9c200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941127e0 con 0x7f4e94107650 2024-03-16T10:07:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e9419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e9419ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:37652/0 (socket says 172.21.15.39:37652) 2024-03-16T10:07:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 -- 172.21.15.39:0/3866860516 learned_addr learned my addr 172.21.15.39:0/3866860516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e99800700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e94102580 0x7f4e9419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:12.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 -- 172.21.15.39:0/3866860516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e94107650 msgr2=0x7f4e94198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e94107650 0x7f4e94198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 -- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e94102580 msgr2=0x7f4e9419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e94102580 0x7f4e9419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e941990e0 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.498+0000 7f4e98e00700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e9419ecb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e80005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.499+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80027070 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.499+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e80022470 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.499+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80017630 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.499+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e94199370 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.499+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e941a5640 con 0x7f4e94102f30 2024-03-16T10:07:12.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.500+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e940629e0 con 0x7f4e94102f30 2024-03-16T10:07:12.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.502+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4e80009d10 con 0x7f4e94102f30 2024-03-16T10:07:12.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.503+0000 7f4e92000700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4e7c0a29a0 0x7f4e7c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:12.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4e800cfeb0 con 0x7f4e94102f30 2024-03-16T10:07:12.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e99800700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4e7c0a29a0 0x7f4e7c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e92000700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] conn(0x7f4e7c0ad660 0x7f4e7c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e8002be50 con 0x7f4e7c0ad660 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4e800d0260 con 0x7f4e94102f30 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.504+0000 7f4e9a200700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] conn(0x7f4e7c0ad660 0x7f4e7c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.505+0000 7f4e99800700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4e7c0a29a0 0x7f4e7c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e941071e0 tx=0x7f4e88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.505+0000 7f4e9a200700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] conn(0x7f4e7c0ad660 0x7f4e7c0afaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:12.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.506+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== osd.6 v2:172.21.15.39:6818/2687294555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e8002be50 con 0x7f4e7c0ad660 2024-03-16T10:07:12.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.525+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 --> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e94062de0 con 0x7f4e7c0ad660 2024-03-16T10:07:12.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.525+0000 7f4e92000700 1 -- 172.21.15.39:0/3866860516 <== osd.6 v2:172.21.15.39:6818/2687294555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e94062de0 con 0x7f4e7c0ad660 2024-03-16T10:07:12.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] conn(0x7f4e7c0ad660 msgr2=0x7f4e7c0afaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] conn(0x7f4e7c0ad660 0x7f4e7c0afaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:12.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4e7c0a29a0 msgr2=0x7f4e7c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4e7c0a29a0 0x7f4e7c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e941071e0 tx=0x7f4e88009260 comp rx=0 tx=0).stop 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 msgr2=0x7f4e9419ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 --2- 172.21.15.39:0/3866860516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e94102f30 0x7f4e9419ecb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e80005590 comp rx=0 tx=0).stop 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9a200700 1 -- 172.21.15.39:0/3866860516 reap_dead start 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 shutdown_connections 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 >> 172.21.15.39:0/3866860516 conn(0x7f4e94079430 msgr2=0x7f4e94100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 shutdown_connections 2024-03-16T10:07:12.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:12.526+0000 7f4e9c200700 1 -- 172.21.15.39:0/3866860516 wait complete. 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-16T10:07:13.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-16T10:07:13.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:12 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]: dispatch 2024-03-16T10:07:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-16T10:07:13.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-16T10:07:13.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:12 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-03-16T10:07:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:12 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-16T10:07:13.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 -- 172.21.15.39:0/124207419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 msgr2=0x7fdaf81028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 --2- 172.21.15.39:0/124207419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf81028f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009990 tx=0x7fdaec009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 -- 172.21.15.39:0/124207419 shutdown_connections 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 --2- 172.21.15.39:0/124207419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaf81075f0 0x7fdaf810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 --2- 172.21.15.39:0/124207419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf81028f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 --2- 172.21.15.39:0/124207419 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaf8102ec0 0x7fdaf8106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 -- 172.21.15.39:0/124207419 >> 172.21.15.39:0/124207419 conn(0x7fdaf80fdd80 msgr2=0x7fdaf81001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 -- 172.21.15.39:0/124207419 shutdown_connections 2024-03-16T10:07:13.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.424+0000 7fdaff600700 1 -- 172.21.15.39:0/124207419 wait complete. 2024-03-16T10:07:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 Processor -- start 2024-03-16T10:07:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 -- start start 2024-03-16T10:07:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf80745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:13.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaf8102ec0 0x7fdaf8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaf81075f0 0x7fdaf8075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112620 con 0x7fdaf81075f0 2024-03-16T10:07:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81124a0 con 0x7fdaf8102510 2024-03-16T10:07:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.426+0000 7fdaff600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81127a0 con 0x7fdaf8102ec0 2024-03-16T10:07:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf80745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf80745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:37670/0 (socket says 172.21.15.39:37670) 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 -- 172.21.15.39:0/2259282932 learned_addr learned my addr 172.21.15.39:0/2259282932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdaf7e00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaf8102ec0 0x7fdaf8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafd600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaf81075f0 0x7fdaf8075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 -- 172.21.15.39:0/2259282932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaf8102ec0 msgr2=0x7fdaf8074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdaf8102ec0 0x7fdaf8074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 -- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaf81075f0 msgr2=0x7fdaf8075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaf81075f0 0x7fdaf8075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a6fb0 con 0x7fdaf8102510 2024-03-16T10:07:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.427+0000 7fdafcc00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf80745d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005910 tx=0x7fdaec0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.428+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec027070 con 0x7fdaf8102510 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.428+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec00b6c0 con 0x7fdaf8102510 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.428+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec017650 con 0x7fdaf8102510 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.428+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a7240 con 0x7fdaf8102510 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.428+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a76f0 con 0x7fdaf8102510 2024-03-16T10:07:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.429+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdaf80629e0 con 0x7fdaf8102510 2024-03-16T10:07:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.432+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fdaec02a070 con 0x7fdaf8102510 2024-03-16T10:07:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf5600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdae00a29a0 0x7fdae00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fdaec0d0580 con 0x7fdaf8102510 2024-03-16T10:07:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf7e00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdae00a29a0 0x7fdae00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf5600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] conn(0x7fdae00ada80 0x7fdae00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaec00fe50 con 0x7fdae00ada80 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fdaec0d09d0 con 0x7fdaf8102510 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.434+0000 7fdafd600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] conn(0x7fdae00ada80 0x7fdae00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.435+0000 7fdaf7e00700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdae00a29a0 0x7fdae00a4e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8108b60 tx=0x7fdae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.435+0000 7fdafd600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] conn(0x7fdae00ada80 0x7fdae00aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.436+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== osd.1 v2:172.21.15.105:6800/3759059886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdaec00fe50 con 0x7fdae00ada80 2024-03-16T10:07:13.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.462+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 --> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdaf8062e60 con 0x7fdae00ada80 2024-03-16T10:07:13.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.463+0000 7fdaf5600700 1 -- 172.21.15.39:0/2259282932 <== osd.1 v2:172.21.15.105:6800/3759059886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdaf8062e60 con 0x7fdae00ada80 2024-03-16T10:07:13.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.463+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] conn(0x7fdae00ada80 msgr2=0x7fdae00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.463+0000 7fdaff600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] conn(0x7fdae00ada80 0x7fdae00aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdae00a29a0 msgr2=0x7fdae00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdae00a29a0 0x7fdae00a4e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8108b60 tx=0x7fdae4009260 comp rx=0 tx=0).stop 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 msgr2=0x7fdaf80745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 --2- 172.21.15.39:0/2259282932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8102510 0x7fdaf80745d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005910 tx=0x7fdaec0047b0 comp rx=0 tx=0).stop 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdafd600700 1 -- 172.21.15.39:0/2259282932 reap_dead start 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 shutdown_connections 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 >> 172.21.15.39:0/2259282932 conn(0x7fdaf80fdd80 msgr2=0x7fdaf8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 shutdown_connections 2024-03-16T10:07:13.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:13.464+0000 7fdaff600700 1 -- 172.21.15.39:0/2259282932 wait complete. 2024-03-16T10:07:14.161 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-16T10:07:14.161 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-16T10:07:14.161 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-16T10:07:14.162 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:13 smithi039 ceph-mon[31510]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-16T10:07:14.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:13 smithi194 ceph-mon[36445]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6, 5, 9]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-16T10:07:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:13 smithi105 ceph-mon[36559]: osdmap e91: 12 total, 12 up, 12 in 2024-03-16T10:07:14.876 INFO:teuthology.orchestra.run.smithi039.stdout:34359738455 2024-03-16T10:07:14.877 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-16T10:07:15.040 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:14 smithi039 ceph-mon[31510]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T10:07:15.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:14 smithi194 ceph-mon[36445]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T10:07:15.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:14 smithi105 ceph-mon[36559]: osdmap e92: 12 total, 12 up, 12 in 2024-03-16T10:07:15.614 INFO:teuthology.orchestra.run.smithi039.stdout:223338299440 2024-03-16T10:07:15.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-16T10:07:16.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:15 smithi039 ceph-mon[31510]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:07:16.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:15 smithi039 ceph-mon[31510]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T10:07:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:15 smithi194 ceph-mon[36445]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:07:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:15 smithi194 ceph-mon[36445]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T10:07:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:15 smithi105 ceph-mon[36559]: pgmap v353: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:07:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:15 smithi105 ceph-mon[36559]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-16T10:07:16.808 INFO:teuthology.orchestra.run.smithi039.stdout:55834574932 2024-03-16T10:07:16.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-16T10:07:18.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:17 smithi194 ceph-mon[36445]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s 2024-03-16T10:07:18.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:17 smithi039 ceph-mon[31510]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s 2024-03-16T10:07:18.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:17 smithi105 ceph-mon[36559]: pgmap v354: 129 pgs: 3 peering, 126 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s 2024-03-16T10:07:18.969 INFO:teuthology.orchestra.run.smithi039.stdout:201863462968 2024-03-16T10:07:18.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 -- 172.21.15.39:0/880563957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 msgr2=0x7f8408108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 --2- 172.21.15.39:0/880563957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 0x7f8408108700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0099c0 tx=0x7f83fc009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 -- 172.21.15.39:0/880563957 shutdown_connections 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 --2- 172.21.15.39:0/880563957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f840806a920 0x7f840810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 --2- 172.21.15.39:0/880563957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 0x7f8408108700 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 --2- 172.21.15.39:0/880563957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8408069f80 0x7f840806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.445+0000 7f8410800700 1 -- 172.21.15.39:0/880563957 >> 172.21.15.39:0/880563957 conn(0x7f8408078100 msgr2=0x7f8408078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.446+0000 7f8410800700 1 -- 172.21.15.39:0/880563957 shutdown_connections 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.446+0000 7f8410800700 1 -- 172.21.15.39:0/880563957 wait complete. 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.447+0000 7f8410800700 1 Processor -- start 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.447+0000 7f8410800700 1 -- start start 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8408069f80 0x7f840819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:19.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 0x7f840819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 0x7f84081988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112560 con 0x7f8408069f80 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84081123e0 con 0x7f840806a920 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f8410800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f84081126e0 con 0x7f8408108320 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 0x7f840819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840de00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8408069f80 0x7f840819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:19.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 0x7f840819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:37676/0 (socket says 172.21.15.39:37676) 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 -- 172.21.15.39:0/4002574243 learned_addr learned my addr 172.21.15.39:0/4002574243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840e800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 0x7f84081988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 -- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 msgr2=0x7f84081988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8408108320 0x7f84081988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 -- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8408069f80 msgr2=0x7f840819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8408069f80 0x7f840819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840d400700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8408199080 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.448+0000 7f840de00700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8408069f80 0x7f840819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f840d400700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 0x7f840819ec10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83f800e9a0 tx=0x7f83f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f800c9c0 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f800cb60 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8408199370 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8018660 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.449+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a5530 con 0x7f840806a920 2024-03-16T10:07:19.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.450+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8408062a60 con 0x7f840806a920 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.452+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f83f800cd10 con 0x7f840806a920 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f8402a00700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f83f00a2920 0x7f83f00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f83f80cf5c0 con 0x7f840806a920 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f840de00700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f83f00a2920 0x7f83f00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f8402a00700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] conn(0x7f83f00ada00 0x7f83f00afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83f801bec0 con 0x7f83f00ada00 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.455+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f83f80cf970 con 0x7f840806a920 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.456+0000 7f840e800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] conn(0x7f83f00ada00 0x7f83f00afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.456+0000 7f840de00700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f83f00a2920 0x7f83f00a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83fc000c00 tx=0x7f83fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:19.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.456+0000 7f840e800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] conn(0x7f83f00ada00 0x7f83f00afea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.458+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== osd.8 v2:172.21.15.194:6816/4066786561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83f801bec0 con 0x7f83f00ada00 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.475+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 --> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8408062ee0 con 0x7f83f00ada00 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.475+0000 7f8402a00700 1 -- 172.21.15.39:0/4002574243 <== osd.8 v2:172.21.15.194:6816/4066786561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8408062ee0 con 0x7f83f00ada00 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] conn(0x7f83f00ada00 msgr2=0x7f83f00afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] conn(0x7f83f00ada00 0x7f83f00afea0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f83f00a2920 msgr2=0x7f83f00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f83f00a2920 0x7f83f00a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83fc000c00 tx=0x7f83fc005fb0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 msgr2=0x7f840819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 --2- 172.21.15.39:0/4002574243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f840806a920 0x7f840819ec10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83f800e9a0 tx=0x7f83f800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f840e800700 1 -- 172.21.15.39:0/4002574243 reap_dead start 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 shutdown_connections 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 >> 172.21.15.39:0/4002574243 conn(0x7f8408078100 msgr2=0x7f84080ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.476+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 shutdown_connections 2024-03-16T10:07:19.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:16.477+0000 7f8410800700 1 -- 172.21.15.39:0/4002574243 wait complete. 2024-03-16T10:07:20.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:19 smithi039 ceph-mon[31510]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-16T10:07:20.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:19 smithi105 ceph-mon[36559]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-16T10:07:20.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:19 smithi194 ceph-mon[36445]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-16T10:07:20.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 -- 172.21.15.39:0/3400264315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14241074f0 msgr2=0x7f142410f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 --2- 172.21.15.39:0/3400264315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14241074f0 0x7f142410f0c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b0a0 tx=0x7f141c00b3b0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 -- 172.21.15.39:0/3400264315 shutdown_connections 2024-03-16T10:07:20.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 --2- 172.21.15.39:0/3400264315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14241074f0 0x7f142410f0c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 --2- 172.21.15.39:0/3400264315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 0x7f14241027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 --2- 172.21.15.39:0/3400264315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1424102dc0 0x7f1424106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 -- 172.21.15.39:0/3400264315 >> 172.21.15.39:0/3400264315 conn(0x7f14240fdca0 msgr2=0x7f14241000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:20.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 -- 172.21.15.39:0/3400264315 shutdown_connections 2024-03-16T10:07:20.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.533+0000 7f142a400700 1 -- 172.21.15.39:0/3400264315 wait complete. 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 Processor -- start 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 -- start start 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 0x7f14241a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1424102dc0 0x7f14241a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14241074f0 0x7f142419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f14241124e0 con 0x7f1424102410 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1424112360 con 0x7f1424102dc0 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.535+0000 7f142a400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1424112660 con 0x7f14241074f0 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1429000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 0x7f14241a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1424102dc0 0x7f14241a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1429000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 0x7f14241a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33576/0 (socket says 172.21.15.39:33576) 2024-03-16T10:07:20.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1429000700 1 -- 172.21.15.39:0/1659228135 learned_addr learned my addr 172.21.15.39:0/1659228135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:20.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1429a00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14241074f0 0x7f142419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:20.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 -- 172.21.15.39:0/1659228135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14241074f0 msgr2=0x7f142419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14241074f0 0x7f142419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 -- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 msgr2=0x7f14241a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1424102410 0x7f14241a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142419b0b0 con 0x7f1424102dc0 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.536+0000 7f1423e00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1424102dc0 0x7f14241a0cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f142406ac20 tx=0x7f141000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.537+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410016070 con 0x7f1424102dc0 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.537+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141000fd70 con 0x7f1424102dc0 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.537+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142419b3a0 con 0x7f1424102dc0 2024-03-16T10:07:20.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.537+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141000e3b0 con 0x7f1424102dc0 2024-03-16T10:07:20.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.537+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14241a7660 con 0x7f1424102dc0 2024-03-16T10:07:20.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.538+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14240ff960 con 0x7f1424102dc0 2024-03-16T10:07:20.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.540+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f1410019070 con 0x7f1424102dc0 2024-03-16T10:07:20.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.541+0000 7f1421600700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f140c0a29a0 0x7f140c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:20.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.541+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f14100ced30 con 0x7f1424102dc0 2024-03-16T10:07:20.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.542+0000 7f1429000700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f140c0a29a0 0x7f140c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:20.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.542+0000 7f1421600700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] conn(0x7f140c0ada80 0x7f140c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:20.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.542+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f140c0b05f0 con 0x7f140c0ada80 2024-03-16T10:07:20.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.542+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f14100cf0e0 con 0x7f1424102dc0 2024-03-16T10:07:20.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.542+0000 7f1429a00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] conn(0x7f140c0ada80 0x7f140c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:20.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.543+0000 7f1429000700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f140c0a29a0 0x7f140c0a4e60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f142419ba90 tx=0x7f1418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:20.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.543+0000 7f1429a00700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] conn(0x7f140c0ada80 0x7f140c0aff20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:20.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.544+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== osd.4 v2:172.21.15.105:6808/45639064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f140c0b05f0 con 0x7f140c0ada80 2024-03-16T10:07:20.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.565+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 --> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14241038f0 con 0x7f140c0ada80 2024-03-16T10:07:20.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f1421600700 1 -- 172.21.15.39:0/1659228135 <== osd.4 v2:172.21.15.105:6808/45639064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f14241038f0 con 0x7f140c0ada80 2024-03-16T10:07:20.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] conn(0x7f140c0ada80 msgr2=0x7f140c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] conn(0x7f140c0ada80 0x7f140c0aff20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:20.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f140c0a29a0 msgr2=0x7f140c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f140c0a29a0 0x7f140c0a4e60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f142419ba90 tx=0x7f1418009260 comp rx=0 tx=0).stop 2024-03-16T10:07:20.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1424102dc0 msgr2=0x7f14241a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f142a400700 1 --2- 172.21.15.39:0/1659228135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1424102dc0 0x7f14241a0cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f142406ac20 tx=0x7f141000dc30 comp rx=0 tx=0).stop 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.566+0000 7f1429a00700 1 -- 172.21.15.39:0/1659228135 reap_dead start 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.567+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 shutdown_connections 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.567+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 >> 172.21.15.39:0/1659228135 conn(0x7f14240fdca0 msgr2=0x7f142410d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.567+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 shutdown_connections 2024-03-16T10:07:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:20.567+0000 7f142a400700 1 -- 172.21.15.39:0/1659228135 wait complete. 2024-03-16T10:07:20.597 INFO:teuthology.orchestra.run.smithi039.stdout:257698037804 2024-03-16T10:07:20.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-16T10:07:21.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:21 smithi039 ceph-mon[31510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:07:21.263 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:21 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:07:21.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:21 smithi105 ceph-mon[36559]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:07:21.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:21 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:07:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:21 smithi194 ceph-mon[36445]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:07:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:21 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:07:22.231 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:22 smithi039 ceph-mon[31510]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-03-16T10:07:22.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:22 smithi105 ceph-mon[36559]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-03-16T10:07:22.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:22 smithi194 ceph-mon[36445]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-03-16T10:07:23.420 INFO:teuthology.orchestra.run.smithi039.stdout:146028888133 2024-03-16T10:07:23.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.543+0000 7f311c000700 1 -- 172.21.15.39:0/1376787279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114069f80 msgr2=0x7f311406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.543+0000 7f311c000700 1 --2- 172.21.15.39:0/1376787279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114069f80 0x7f311406a3e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31000099c0 tx=0x7f3100009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 -- 172.21.15.39:0/1376787279 shutdown_connections 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 --2- 172.21.15.39:0/1376787279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f311406a920 0x7f311410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 --2- 172.21.15.39:0/1376787279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3114108320 0x7f3114108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 --2- 172.21.15.39:0/1376787279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114069f80 0x7f311406a3e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 -- 172.21.15.39:0/1376787279 >> 172.21.15.39:0/1376787279 conn(0x7f3114078100 msgr2=0x7f3114078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 -- 172.21.15.39:0/1376787279 shutdown_connections 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.544+0000 7f311c000700 1 -- 172.21.15.39:0/1376787279 wait complete. 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.545+0000 7f311c000700 1 Processor -- start 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 -- start start 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3114069f80 0x7f31141012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 0x7f31141017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:23.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114108320 0x7f3114101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31141125d0 con 0x7f3114069f80 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3114112450 con 0x7f311406a920 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311c000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3114112750 con 0x7f3114108320 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f3118c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 0x7f31141017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f3119600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3114069f80 0x7f31141012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.546+0000 7f311a000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114108320 0x7f3114101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 0x7f31141017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54944/0 (socket says 172.21.15.39:54944) 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 -- 172.21.15.39:0/67876505 learned_addr learned my addr 172.21.15.39:0/67876505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 -- 172.21.15.39:0/67876505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114108320 msgr2=0x7f3114101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114108320 0x7f3114101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 -- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3114069f80 msgr2=0x7f31141012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3114069f80 0x7f31141012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31141a7030 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f311a000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3114108320 0x7f3114101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.547+0000 7f3118c00700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 0x7f31141017e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3100006010 tx=0x7f3100003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.548+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100003e50 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.548+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31000055a0 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.548+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310000fe30 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.548+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141a72c0 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.548+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141a7770 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.549+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31140629e0 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.551+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f310000f4f0 con 0x7f311406a920 2024-03-16T10:07:23.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.552+0000 7f3112000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f30fc0a2b40 0x7f30fc0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.552+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f31000d00f0 con 0x7f311406a920 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f3119600700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f30fc0a2b40 0x7f30fc0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f3112000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] conn(0x7f30fc0adc20 0x7f30fc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3100021e80 con 0x7f30fc0adc20 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f31000d04a0 con 0x7f311406a920 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f311a000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] conn(0x7f30fc0adc20 0x7f30fc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.553+0000 7f3119600700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f30fc0a2b40 0x7f30fc0a5000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3108009be0 tx=0x7f3108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.554+0000 7f311a000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] conn(0x7f30fc0adc20 0x7f30fc0b00c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.555+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== osd.9 v2:172.21.15.39:6826/2744339033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3100021e80 con 0x7f30fc0adc20 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.574+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 --> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3114062de0 con 0x7f30fc0adc20 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.574+0000 7f3112000700 1 -- 172.21.15.39:0/67876505 <== osd.9 v2:172.21.15.39:6826/2744339033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3114062de0 con 0x7f30fc0adc20 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] conn(0x7f30fc0adc20 msgr2=0x7f30fc0b00c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] conn(0x7f30fc0adc20 0x7f30fc0b00c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f30fc0a2b40 msgr2=0x7f30fc0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f30fc0a2b40 0x7f30fc0a5000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3108009be0 tx=0x7f3108009260 comp rx=0 tx=0).stop 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 msgr2=0x7f31141017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 --2- 172.21.15.39:0/67876505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f311406a920 0x7f31141017e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3100006010 tx=0x7f3100003a90 comp rx=0 tx=0).stop 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311a000700 1 -- 172.21.15.39:0/67876505 reap_dead start 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 shutdown_connections 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 >> 172.21.15.39:0/67876505 conn(0x7f3114078100 msgr2=0x7f31140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 shutdown_connections 2024-03-16T10:07:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:22.575+0000 7f311c000700 1 -- 172.21.15.39:0/67876505 wait complete. 2024-03-16T10:07:24.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:24 smithi105 ceph-mon[36559]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 1 objects/s recovering 2024-03-16T10:07:24.401 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:24 smithi039 ceph-mon[31510]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 1 objects/s recovering 2024-03-16T10:07:24.450 INFO:teuthology.orchestra.run.smithi039.stdout:292057776167 2024-03-16T10:07:24.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-16T10:07:24.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:24 smithi194 ceph-mon[36445]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 12 B/s, 1 objects/s recovering 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 -- 172.21.15.39:0/3734537090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 msgr2=0x7f6d6406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3734537090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d6406a3e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009990 tx=0x7f6d50009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 -- 172.21.15.39:0/3734537090 shutdown_connections 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3734537090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d6406a920 0x7f6d6410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3734537090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d64108320 0x7f6d64108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3734537090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d6406a3e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 -- 172.21.15.39:0/3734537090 >> 172.21.15.39:0/3734537090 conn(0x7f6d64078100 msgr2=0x7f6d64078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 -- 172.21.15.39:0/3734537090 shutdown_connections 2024-03-16T10:07:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.655+0000 7f6d6c400700 1 -- 172.21.15.39:0/3734537090 wait complete. 2024-03-16T10:07:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.657+0000 7f6d6c400700 1 Processor -- start 2024-03-16T10:07:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.657+0000 7f6d6c400700 1 -- start start 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d641012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d6406a920 0x7f6d641017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d64108320 0x7f6d64101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641125d0 con 0x7f6d6406a920 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64112450 con 0x7f6d64069f80 2024-03-16T10:07:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6c400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64112750 con 0x7f6d64108320 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d641012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6a400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d64108320 0x7f6d64101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d641012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54956/0 (socket says 172.21.15.39:54956) 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 -- 172.21.15.39:0/3398819332 learned_addr learned my addr 172.21.15.39:0/3398819332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 -- 172.21.15.39:0/3398819332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d64108320 msgr2=0x7f6d64101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d64108320 0x7f6d64101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 -- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d6406a920 msgr2=0x7f6d641017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d6406a920 0x7f6d641017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d69a00700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d641a6f90 con 0x7f6d64069f80 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.658+0000 7f6d6a400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6d64108320 0x7f6d64101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d69a00700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d641012a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9a0 tx=0x7f6d5800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800cbd0 con 0x7f6d64069f80 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5800cd70 con 0x7f6d64069f80 2024-03-16T10:07:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58018730 con 0x7f6d64069f80 2024-03-16T10:07:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d641a7280 con 0x7f6d64069f80 2024-03-16T10:07:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.659+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641a7770 con 0x7f6d64069f80 2024-03-16T10:07:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.660+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d640629e0 con 0x7f6d64069f80 2024-03-16T10:07:24.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.664+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6d58010430 con 0x7f6d64069f80 2024-03-16T10:07:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.665+0000 7f6d62000700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6d4c0a29a0 0x7f6d4c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.665+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f6d580cf0e0 con 0x7f6d64069f80 2024-03-16T10:07:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.666+0000 7f6d69000700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6d4c0a29a0 0x7f6d4c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.666+0000 7f6d62000700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] conn(0x7f6d4c0ada80 0x7f6d4c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.666+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d58012ec0 con 0x7f6d4c0ada80 2024-03-16T10:07:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.666+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6d580d6050 con 0x7f6d64069f80 2024-03-16T10:07:24.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.666+0000 7f6d6a400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] conn(0x7f6d4c0ada80 0x7f6d4c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:24.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.667+0000 7f6d69000700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6d4c0a29a0 0x7f6d4c0a4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098e0 tx=0x7f6d50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:24.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.667+0000 7f6d6a400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] conn(0x7f6d4c0ada80 0x7f6d4c0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.668+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== osd.3 v2:172.21.15.39:6810/2181780282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d58012ec0 con 0x7f6d4c0ada80 2024-03-16T10:07:24.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.686+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 --> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d64062eb0 con 0x7f6d4c0ada80 2024-03-16T10:07:24.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.687+0000 7f6d62000700 1 -- 172.21.15.39:0/3398819332 <== osd.3 v2:172.21.15.39:6810/2181780282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d64062eb0 con 0x7f6d4c0ada80 2024-03-16T10:07:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.687+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] conn(0x7f6d4c0ada80 msgr2=0x7f6d4c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.687+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] conn(0x7f6d4c0ada80 0x7f6d4c0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6d4c0a29a0 msgr2=0x7f6d4c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6d4c0a29a0 0x7f6d4c0a4e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d500098e0 tx=0x7f6d50005fb0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 msgr2=0x7f6d641012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 --2- 172.21.15.39:0/3398819332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d64069f80 0x7f6d641012a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800e9a0 tx=0x7f6d5800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6a400700 1 -- 172.21.15.39:0/3398819332 reap_dead start 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 shutdown_connections 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 >> 172.21.15.39:0/3398819332 conn(0x7f6d64078100 msgr2=0x7f6d640ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 shutdown_connections 2024-03-16T10:07:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:24.688+0000 7f6d6c400700 1 -- 172.21.15.39:0/3398819332 wait complete. 2024-03-16T10:07:26.161 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:26 smithi039 ceph-mon[31510]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-03-16T10:07:26.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:26 smithi105 ceph-mon[36559]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-03-16T10:07:26.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:26 smithi194 ceph-mon[36445]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 10 B/s, 0 objects/s recovering 2024-03-16T10:07:26.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 -- 172.21.15.39:0/3116304436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec06b420 msgr2=0x7f0bec106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 --2- 172.21.15.39:0/3116304436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec06b420 0x7f0bec106f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80099c0 tx=0x7f0bd8009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 -- 172.21.15.39:0/3116304436 shutdown_connections 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 --2- 172.21.15.39:0/3116304436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec107650 0x7f0bec10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 --2- 172.21.15.39:0/3116304436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bec06aa70 0x7f0bec06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 --2- 172.21.15.39:0/3116304436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec06b420 0x7f0bec106f20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 -- 172.21.15.39:0/3116304436 >> 172.21.15.39:0/3116304436 conn(0x7f0bec077220 msgr2=0x7f0bec077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 -- 172.21.15.39:0/3116304436 shutdown_connections 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.517+0000 7f0bf4400700 1 -- 172.21.15.39:0/3116304436 wait complete. 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.518+0000 7f0bf4400700 1 Processor -- start 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 -- start start 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bec06aa70 0x7f0bec1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 0x7f0bec101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec107650 0x7f0bec101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec1125a0 con 0x7f0bec06aa70 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec112420 con 0x7f0bec06b420 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.519+0000 7f0bf4400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0bec112720 con 0x7f0bec107650 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 0x7f0bec101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bec06aa70 0x7f0bec1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 0x7f0bec101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54974/0 (socket says 172.21.15.39:54974) 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 -- 172.21.15.39:0/712763027 learned_addr learned my addr 172.21.15.39:0/712763027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf2400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec107650 0x7f0bec101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 -- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec107650 msgr2=0x7f0bec101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec107650 0x7f0bec101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 -- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bec06aa70 msgr2=0x7f0bec1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bec06aa70 0x7f0bec1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bec1a6fc0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf2400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bec107650 0x7f0bec101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.520+0000 7f0bf1000700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 0x7f0bec101810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800b580 tx=0x7f0bd8003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.521+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8009e60 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.521+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bd8003680 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.521+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800fdf0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.521+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec1a7250 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.521+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bec1a76a0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.522+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0bec0629e0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.524+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0bd800f4f0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.525+0000 7f0bea000700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bd40a29a0 0x7f0bd40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f0bd80cfce0 con 0x7f0bec06b420 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bf1a00700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bd40a29a0 0x7f0bd40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bea000700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] conn(0x7f0bd40ada80 0x7f0bd40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bd40b05f0 con 0x7f0bd40ada80 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0bd80d2300 con 0x7f0bec06b420 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.526+0000 7f0bf2400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] conn(0x7f0bd40ada80 0x7f0bd40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.527+0000 7f0bf1a00700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bd40a29a0 0x7f0bd40a4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0be0005950 tx=0x7f0be000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.527+0000 7f0bf2400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] conn(0x7f0bd40ada80 0x7f0bd40aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.528+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== osd.11 v2:172.21.15.194:6824/254372384 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd40b05f0 con 0x7f0bd40ada80 2024-03-16T10:07:26.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.547+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 --> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0bec062e60 con 0x7f0bd40ada80 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.547+0000 7f0bea000700 1 -- 172.21.15.39:0/712763027 <== osd.11 v2:172.21.15.194:6824/254372384 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0bec062e60 con 0x7f0bd40ada80 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.547+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] conn(0x7f0bd40ada80 msgr2=0x7f0bd40aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.547+0000 7f0bf4400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] conn(0x7f0bd40ada80 0x7f0bd40aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bd40a29a0 msgr2=0x7f0bd40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0bd40a29a0 0x7f0bd40a4e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0be0005950 tx=0x7f0be000b410 comp rx=0 tx=0).stop 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 msgr2=0x7f0bec101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 --2- 172.21.15.39:0/712763027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bec06b420 0x7f0bec101810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800b580 tx=0x7f0bd8003e00 comp rx=0 tx=0).stop 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf2400700 1 -- 172.21.15.39:0/712763027 reap_dead start 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 shutdown_connections 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 >> 172.21.15.39:0/712763027 conn(0x7f0bec077220 msgr2=0x7f0bec0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 shutdown_connections 2024-03-16T10:07:26.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:25.548+0000 7f0bf4400700 1 -- 172.21.15.39:0/712763027 wait complete. 2024-03-16T10:07:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 -- 172.21.15.39:0/2072083185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104500 msgr2=0x7fa070104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 --2- 172.21.15.39:0/2072083185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104500 0x7fa070104980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0099c0 tx=0x7fa05c009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 -- 172.21.15.39:0/2072083185 shutdown_connections 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 --2- 172.21.15.39:0/2072083185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070104ec0 0x7fa070111380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 --2- 172.21.15.39:0/2072083185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa07010a5b0 0x7fa07010a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 --2- 172.21.15.39:0/2072083185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104500 0x7fa070104980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 -- 172.21.15.39:0/2072083185 >> 172.21.15.39:0/2072083185 conn(0x7fa0700fffd0 msgr2=0x7fa0701023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 -- 172.21.15.39:0/2072083185 shutdown_connections 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.549+0000 7fa075000700 1 -- 172.21.15.39:0/2072083185 wait complete. 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.550+0000 7fa075000700 1 Processor -- start 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.550+0000 7fa075000700 1 -- start start 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070104500 0x7fa0701a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 0x7fa0701a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa07010a5b0 0x7fa07019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa070114840 con 0x7fa070104500 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701146c0 con 0x7fa070104ec0 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa075000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701149c0 con 0x7fa07010a5b0 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa06d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 0x7fa0701a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa06d400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 0x7fa0701a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:55004/0 (socket says 172.21.15.39:55004) 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa06d400700 1 -- 172.21.15.39:0/2899145180 learned_addr learned my addr 172.21.15.39:0/2899145180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa06de00700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070104500 0x7fa0701a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.551+0000 7fa06e800700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa07010a5b0 0x7fa07019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 -- 172.21.15.39:0/2899145180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa07010a5b0 msgr2=0x7fa07019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa07010a5b0 0x7fa07019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 -- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070104500 msgr2=0x7fa0701a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070104500 0x7fa0701a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07019b180 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06e800700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa07010a5b0 0x7fa07019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa06d400700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 0x7fa0701a0da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005910 tx=0x7fa05c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c027070 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c003680 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0173f0 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.552+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07019b410 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.553+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0701a7730 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.554+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa070062a60 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.556+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fa05c00fcd0 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.557+0000 7fa066a00700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa0540a29a0 0x7fa0540a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.557+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fa05c0cfab0 con 0x7fa070104ec0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.557+0000 7fa06de00700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa0540a29a0 0x7fa0540a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.558+0000 7fa066a00700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] conn(0x7fa0540ada00 0x7fa0540afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.558+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa05c009030 con 0x7fa0540ada00 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.558+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa05c0cfe60 con 0x7fa070104ec0 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.558+0000 7fa06e800700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] conn(0x7fa0540ada00 0x7fa0540afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.558+0000 7fa06de00700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa0540a29a0 0x7fa0540a4e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa07019bb60 tx=0x7fa060009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.559+0000 7fa06e800700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] conn(0x7fa0540ada00 0x7fa0540afea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.559+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== osd.10 v2:172.21.15.105:6824/4246880500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa05c009030 con 0x7fa0540ada00 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.577+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 --> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa070062ee0 con 0x7fa0540ada00 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa066a00700 1 -- 172.21.15.39:0/2899145180 <== osd.10 v2:172.21.15.105:6824/4246880500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa070062ee0 con 0x7fa0540ada00 2024-03-16T10:07:26.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] conn(0x7fa0540ada00 msgr2=0x7fa0540afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] conn(0x7fa0540ada00 0x7fa0540afea0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa0540a29a0 msgr2=0x7fa0540a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa0540a29a0 0x7fa0540a4e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa07019bb60 tx=0x7fa060009260 comp rx=0 tx=0).stop 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 msgr2=0x7fa0701a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.578+0000 7fa075000700 1 --2- 172.21.15.39:0/2899145180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa070104ec0 0x7fa0701a0da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa05c005910 tx=0x7fa05c00f690 comp rx=0 tx=0).stop 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.579+0000 7fa06e800700 1 -- 172.21.15.39:0/2899145180 reap_dead start 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.579+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 shutdown_connections 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.579+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 >> 172.21.15.39:0/2899145180 conn(0x7fa0700fffd0 msgr2=0x7fa070101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.579+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 shutdown_connections 2024-03-16T10:07:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:26.579+0000 7fa075000700 1 -- 172.21.15.39:0/2899145180 wait complete. 2024-03-16T10:07:27.857 INFO:teuthology.orchestra.run.smithi039.stdout:347892351007 2024-03-16T10:07:27.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-16T10:07:28.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.067+0000 7fbe43e00700 1 -- 172.21.15.39:0/2968370012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c1075f0 msgr2=0x7fbe3c10f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.067+0000 7fbe43e00700 1 --2- 172.21.15.39:0/2968370012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c1075f0 0x7fbe3c10f1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b0a0 tx=0x7fbe3400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 -- 172.21.15.39:0/2968370012 shutdown_connections 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 --2- 172.21.15.39:0/2968370012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c1075f0 0x7fbe3c10f1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 --2- 172.21.15.39:0/2968370012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 --2- 172.21.15.39:0/2968370012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c102ec0 0x7fbe3c106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 -- 172.21.15.39:0/2968370012 >> 172.21.15.39:0/2968370012 conn(0x7fbe3c0fdd80 msgr2=0x7fbe3c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:28.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 -- 172.21.15.39:0/2968370012 shutdown_connections 2024-03-16T10:07:28.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.068+0000 7fbe43e00700 1 -- 172.21.15.39:0/2968370012 wait complete. 2024-03-16T10:07:28.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.069+0000 7fbe43e00700 1 Processor -- start 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.069+0000 7fbe43e00700 1 -- start start 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c102ec0 0x7fbe3c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1075f0 0x7fbe3c075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c112630 con 0x7fbe3c1075f0 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c1124b0 con 0x7fbe3c102ec0 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe43e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c1127b0 con 0x7fbe3c102510 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe41400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe41400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c0745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:33714/0 (socket says 172.21.15.39:33714) 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe41400700 1 -- 172.21.15.39:0/3769438167 learned_addr learned my addr 172.21.15.39:0/3769438167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:28.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe40a00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c102ec0 0x7fbe3c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.070+0000 7fbe41e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1075f0 0x7fbe3c075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 -- 172.21.15.39:0/3769438167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c102ec0 msgr2=0x7fbe3c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe3c102ec0 0x7fbe3c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 -- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1075f0 msgr2=0x7fbe3c075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe3c1075f0 0x7fbe3c075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c1ab470 con 0x7fbe3c102510 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe41400700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c0745d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9a0 tx=0x7fbe3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000ca10 con 0x7fbe3c102510 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3000cbb0 con 0x7fbe3c102510 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.071+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30018680 con 0x7fbe3c102510 2024-03-16T10:07:28.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.072+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1ab700 con 0x7fbe3c102510 2024-03-16T10:07:28.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.072+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1abbf0 con 0x7fbe3c102510 2024-03-16T10:07:28.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.073+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe3c0629e0 con 0x7fbe3c102510 2024-03-16T10:07:28.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.077+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fbe30010430 con 0x7fbe3c102510 2024-03-16T10:07:28.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.078+0000 7fbe3a000700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbe240a29a0 0x7fbe240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:28.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.078+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fbe300cf940 con 0x7fbe3c102510 2024-03-16T10:07:28.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.079+0000 7fbe40a00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbe240a29a0 0x7fbe240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:28.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.079+0000 7fbe3a000700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] conn(0x7fbe240ada80 0x7fbe240aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:28.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.079+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe3001bec0 con 0x7fbe240ada80 2024-03-16T10:07:28.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.079+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbe300cfcf0 con 0x7fbe3c102510 2024-03-16T10:07:28.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.079+0000 7fbe41e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] conn(0x7fbe240ada80 0x7fbe240aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:28.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.080+0000 7fbe40a00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbe240a29a0 0x7fbe240a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c075d90 tx=0x7fbe28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:28.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.080+0000 7fbe41e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] conn(0x7fbe240ada80 0x7fbe240aff20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:28.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.081+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== osd.2 v2:172.21.15.194:6800/3029390216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe3001bec0 con 0x7fbe240ada80 2024-03-16T10:07:28.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.099+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 --> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe3c062de0 con 0x7fbe240ada80 2024-03-16T10:07:28.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.099+0000 7fbe3a000700 1 -- 172.21.15.39:0/3769438167 <== osd.2 v2:172.21.15.194:6800/3029390216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbe3c062de0 con 0x7fbe240ada80 2024-03-16T10:07:28.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] conn(0x7fbe240ada80 msgr2=0x7fbe240aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] conn(0x7fbe240ada80 0x7fbe240aff20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbe240a29a0 msgr2=0x7fbe240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbe240a29a0 0x7fbe240a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c075d90 tx=0x7fbe28009260 comp rx=0 tx=0).stop 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 msgr2=0x7fbe3c0745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 --2- 172.21.15.39:0/3769438167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe3c102510 0x7fbe3c0745d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9a0 tx=0x7fbe3000ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe41e00700 1 -- 172.21.15.39:0/3769438167 reap_dead start 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 shutdown_connections 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 >> 172.21.15.39:0/3769438167 conn(0x7fbe3c0fdd80 msgr2=0x7fbe3c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 shutdown_connections 2024-03-16T10:07:28.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:28.100+0000 7fbe43e00700 1 -- 172.21.15.39:0/3769438167 wait complete. 2024-03-16T10:07:28.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:28 smithi105 ceph-mon[36559]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:28.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:28 smithi194 ceph-mon[36445]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:28.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:28 smithi039 ceph-mon[31510]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:28.939 INFO:teuthology.orchestra.run.smithi039.stdout:313532612642 2024-03-16T10:07:28.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-16T10:07:29.220 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:29.275 INFO:teuthology.orchestra.run.smithi039.stdout:111669149770 2024-03-16T10:07:29.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-16T10:07:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:29.581 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:29.603 INFO:teuthology.orchestra.run.smithi039.stdout:77309411408 2024-03-16T10:07:29.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-16T10:07:30.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:30 smithi039 ceph-mon[31510]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:30.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:30 smithi105 ceph-mon[36559]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:30.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:30 smithi194 ceph-mon[36445]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:30.717 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:30.721 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:30.839 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:32.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:32 smithi105 ceph-mon[36559]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:32.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:32 smithi194 ceph-mon[36445]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:32.500 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:32 smithi039 ceph-mon[31510]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 9 B/s, 0 objects/s recovering 2024-03-16T10:07:34.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:34 smithi039 ceph-mon[31510]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:34.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:34 smithi105 ceph-mon[36559]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:34.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:34 smithi194 ceph-mon[36445]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:36.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:36 smithi105 ceph-mon[36559]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:07:36.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:36 smithi194 ceph-mon[36445]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:07:36.481 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:36 smithi039 ceph-mon[31510]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:07:36.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.750+0000 7f77a4400700 1 -- 172.21.15.39:0/3727059526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 msgr2=0x7f779c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:36.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.750+0000 7f77a4400700 1 --2- 172.21.15.39:0/3727059526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c106f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77880099c0 tx=0x7f7788009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 -- 172.21.15.39:0/3727059526 shutdown_connections 2024-03-16T10:07:36.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 --2- 172.21.15.39:0/3727059526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c10f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 --2- 172.21.15.39:0/3727059526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 --2- 172.21.15.39:0/3727059526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c106f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 -- 172.21.15.39:0/3727059526 >> 172.21.15.39:0/3727059526 conn(0x7f779c079430 msgr2=0x7f779c079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 -- 172.21.15.39:0/3727059526 shutdown_connections 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.751+0000 7f77a4400700 1 -- 172.21.15.39:0/3727059526 wait complete. 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.752+0000 7f77a4400700 1 Processor -- start 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.752+0000 7f77a4400700 1 -- start start 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f779c112660 con 0x7f779c102f30 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1124e0 con 0x7f779c107650 2024-03-16T10:07:36.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a4400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1127e0 con 0x7f779c102580 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a1a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a2400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a1a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:58912/0 (socket says 172.21.15.39:58912) 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a1000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.753+0000 7f77a1a00700 1 -- 172.21.15.39:0/2028345623 learned_addr learned my addr 172.21.15.39:0/2028345623 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 -- 172.21.15.39:0/2028345623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 msgr2=0x7f779c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 -- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 msgr2=0x7f779c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 -- 172.21.15.39:0/2028345623 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c199180 con 0x7f779c102580 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a2400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f77a1a00700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f779000ef10 tx=0x7f779000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.754+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779000c9c0 con 0x7f779c102580 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.755+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c199410 con 0x7f779c102580 2024-03-16T10:07:36.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.755+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7790019070 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.755+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790007e60 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.755+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1a56e0 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.756+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c068f90 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.758+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7790012050 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.759+0000 7f779a000700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 0x7f77840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.759+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f77900ceef0 con 0x7f779c102580 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.759+0000 7f77a1000700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 0x7f77840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.760+0000 7f77a1000700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 0x7f77840a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f7788005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:36.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:36.763+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7790010040 con 0x7f779c102580 2024-03-16T10:07:37.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.042+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f779c02db70 con 0x7f779c102580 2024-03-16T10:07:37.045 INFO:teuthology.orchestra.run.smithi039.stdout:223338299445 2024-03-16T10:07:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.043+0000 7f779a000700 1 -- 172.21.15.39:0/2028345623 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f779009ba90 con 0x7f779c102580 2024-03-16T10:07:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 msgr2=0x7f77840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 0x7f77840a4de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7788000c00 tx=0x7f7788005fb0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 msgr2=0x7f779c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f779000ef10 tx=0x7f779000c3a0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 shutdown_connections 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77840a2920 0x7f77840a4de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f779c107650 0x7f779c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f779c102580 0x7f779c19e810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 --2- 172.21.15.39:0/2028345623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779c102f30 0x7f779c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 >> 172.21.15.39:0/2028345623 conn(0x7f779c079430 msgr2=0x7f779c100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.046+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 shutdown_connections 2024-03-16T10:07:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:37.047+0000 7f77a4400700 1 -- 172.21.15.39:0/2028345623 wait complete. 2024-03-16T10:07:37.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:37 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2028345623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T10:07:37.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:37 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2028345623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T10:07:37.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:37 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2028345623' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T10:07:38.250 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:38 smithi039 ceph-mon[31510]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:07:38.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:38 smithi105 ceph-mon[36559]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:07:38.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:38 smithi194 ceph-mon[36445]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:07:38.843 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299440 got 223338299445 for osd.7 2024-03-16T10:07:38.844 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 -- 172.21.15.39:0/2469517639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e0108320 msgr2=0x7f49e0108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 --2- 172.21.15.39:0/2469517639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e0108320 0x7f49e0108700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49d00099c0 tx=0x7f49d0009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 -- 172.21.15.39:0/2469517639 shutdown_connections 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 --2- 172.21.15.39:0/2469517639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e006a920 0x7f49e010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 --2- 172.21.15.39:0/2469517639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e0108320 0x7f49e0108700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 --2- 172.21.15.39:0/2469517639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 0x7f49e006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 -- 172.21.15.39:0/2469517639 >> 172.21.15.39:0/2469517639 conn(0x7f49e0078100 msgr2=0x7f49e0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 -- 172.21.15.39:0/2469517639 shutdown_connections 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.053+0000 7f49e6600700 1 -- 172.21.15.39:0/2469517639 wait complete. 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.054+0000 7f49e6600700 1 Processor -- start 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 -- start start 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 0x7f49e01a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e0108320 0x7f49e019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01125b0 con 0x7f49e0069f80 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0112430 con 0x7f49e0108320 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49e6600700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0112730 con 0x7f49e006a920 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.055+0000 7f49dea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47870/0 (socket says 172.21.15.39:47870) 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 -- 172.21.15.39:0/4185086841 learned_addr learned my addr 172.21.15.39:0/4185086841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49df400700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 0x7f49e01a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dfe00700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e0108320 0x7f49e019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 -- 172.21.15.39:0/4185086841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e0108320 msgr2=0x7f49e019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e0108320 0x7f49e019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 -- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 msgr2=0x7f49e01a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 0x7f49e01a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 -- 172.21.15.39:0/4185086841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e019b220 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.056+0000 7f49dea00700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e9a0 tx=0x7f49cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.057+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00c9c0 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.057+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49cc00cb60 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.057+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e019b510 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.057+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc018660 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.057+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e01090a0 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.058+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e0068f90 con 0x7f49e006a920 2024-03-16T10:07:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.060+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f49cc00cd10 con 0x7f49e006a920 2024-03-16T10:07:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.061+0000 7f49d7e00700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 0x7f49c40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.062+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f49cc0cf0b0 con 0x7f49e006a920 2024-03-16T10:07:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.062+0000 7f49df400700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 0x7f49c40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.063+0000 7f49df400700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 0x7f49c40a4e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d0022040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.070+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f49cc09bc50 con 0x7f49e006a920 2024-03-16T10:07:40.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:40 smithi039 ceph-mon[31510]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.328+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f49e0109300 con 0x7f49e006a920 2024-03-16T10:07:40.389 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:40 smithi105 ceph-mon[36559]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:40.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.328+0000 7f49d7e00700 1 -- 172.21.15.39:0/4185086841 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49cc09b550 con 0x7f49e006a920 2024-03-16T10:07:40.389 INFO:teuthology.orchestra.run.smithi039.stdout:201863462973 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 msgr2=0x7f49c40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 0x7f49c40a4e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d0022040 comp rx=0 tx=0).stop 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 msgr2=0x7f49e01a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e9a0 tx=0x7f49cc00ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 shutdown_connections 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f49c40a29a0 0x7f49c40a4e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49e0108320 0x7f49e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49e0069f80 0x7f49e01a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 --2- 172.21.15.39:0/4185086841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e006a920 0x7f49e01a0e40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:40.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 >> 172.21.15.39:0/4185086841 conn(0x7f49e0078100 msgr2=0x7f49e010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:40.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 shutdown_connections 2024-03-16T10:07:40.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:40.331+0000 7f49e6600700 1 -- 172.21.15.39:0/4185086841 wait complete. 2024-03-16T10:07:40.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:40 smithi194 ceph-mon[36445]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:41.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:41 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4185086841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T10:07:41.379 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:41 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4185086841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T10:07:41.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:41 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4185086841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T10:07:42.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 -- 172.21.15.39:0/2746643051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 msgr2=0x7f1b5010f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 --2- 172.21.15.39:0/2746643051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b5010f0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b0d0 tx=0x7f1b4c00b3e0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 -- 172.21.15.39:0/2746643051 shutdown_connections 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 --2- 172.21.15.39:0/2746643051 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b5010f0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 --2- 172.21.15.39:0/2746643051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b501027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 --2- 172.21.15.39:0/2746643051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 0x7f1b50106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 -- 172.21.15.39:0/2746643051 >> 172.21.15.39:0/2746643051 conn(0x7f1b500fdca0 msgr2=0x7f1b501000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 -- 172.21.15.39:0/2746643051 shutdown_connections 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.540+0000 7f1b57200700 1 -- 172.21.15.39:0/2746643051 wait complete. 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 Processor -- start 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 -- start start 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 0x7f1b5019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b501987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50112530 con 0x7f1b50102dc0 2024-03-16T10:07:42.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501123b0 con 0x7f1b50102410 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.541+0000 7f1b57200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501126b0 con 0x7f1b501074f0 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54546/0 (socket says 172.21.15.39:54546) 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 -- 172.21.15.39:0/3952014379 learned_addr learned my addr 172.21.15.39:0/3952014379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b56800700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b501987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55400700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 0x7f1b5019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 -- 172.21.15.39:0/3952014379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 msgr2=0x7f1b501987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b501987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 -- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 msgr2=0x7f1b5019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 0x7f1b5019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 -- 172.21.15.39:0/3952014379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b50199090 con 0x7f1b50102410 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b56800700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b501987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.542+0000 7f1b55e00700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9a0 tx=0x7f1b4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.543+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400ca10 con 0x7f1b50102410 2024-03-16T10:07:42.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.543+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4400cbb0 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.543+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b50199380 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.543+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44018680 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.543+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a5610 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.544+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b501038f0 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.547+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f1b44010430 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.549+0000 7f1b4aa00700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 0x7f1b380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.549+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f1b440cf140 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.549+0000 7f1b55400700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 0x7f1b380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.550+0000 7f1b55400700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 0x7f1b380a4e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b50199f20 tx=0x7f1b40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.553+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b4409bce0 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.819+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1b50199c90 con 0x7f1b50102410 2024-03-16T10:07:42.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.819+0000 7f1b4aa00700 1 -- 172.21.15.39:0/3952014379 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b4409b5e0 con 0x7f1b50102410 2024-03-16T10:07:42.006 INFO:teuthology.orchestra.run.smithi039.stdout:167503724610 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 msgr2=0x7f1b380a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 0x7f1b380a4e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b50199f20 tx=0x7f1b40009260 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 msgr2=0x7f1b5019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400e9a0 tx=0x7f1b4400ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 shutdown_connections 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1b380a29a0 0x7f1b380a4e60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b501074f0 0x7f1b501987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b50102410 0x7f1b5019e690 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 --2- 172.21.15.39:0/3952014379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b50102dc0 0x7f1b5019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 >> 172.21.15.39:0/3952014379 conn(0x7f1b500fdca0 msgr2=0x7f1b500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 shutdown_connections 2024-03-16T10:07:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.822+0000 7f1b57200700 1 -- 172.21.15.39:0/3952014379 wait complete. 2024-03-16T10:07:42.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:42 smithi105 ceph-mon[36559]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:07:42.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:42 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2543463048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T10:07:42.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:42 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3952014379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T10:07:42.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:42 smithi039 ceph-mon[31510]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:07:42.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:42 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2543463048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T10:07:42.387 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:42 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3952014379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T10:07:42.429 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462968 got 201863462973 for osd.6 2024-03-16T10:07:42.430 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:42.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:42 smithi194 ceph-mon[36445]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:07:42.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:42 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2543463048' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T10:07:42.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:42 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3952014379' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T10:07:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 -- 172.21.15.39:0/275683710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 msgr2=0x7feea81086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 --2- 172.21.15.39:0/275683710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea81086e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fee980099c0 tx=0x7fee98009cd0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 -- 172.21.15.39:0/275683710 shutdown_connections 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 --2- 172.21.15.39:0/275683710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 --2- 172.21.15.39:0/275683710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea81086e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 --2- 172.21.15.39:0/275683710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea806a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 -- 172.21.15.39:0/275683710 >> 172.21.15.39:0/275683710 conn(0x7feea8077f80 msgr2=0x7feea8078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 -- 172.21.15.39:0/275683710 shutdown_connections 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.002+0000 7feead800700 1 -- 172.21.15.39:0/275683710 wait complete. 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.004+0000 7feead800700 1 Processor -- start 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.004+0000 7feead800700 1 -- start start 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea81015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea8101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea8102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feea8112600 con 0x7feea8069f80 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feea8112480 con 0x7feea8108300 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feead800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feea8112780 con 0x7feea806a940 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feea6a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea81015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feea6000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea8101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feea7400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea8102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feea6a00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea81015f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52650/0 (socket says 172.21.15.39:52650) 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.005+0000 7feea6a00700 1 -- 172.21.15.39:0/2543463048 learned_addr learned my addr 172.21.15.39:0/2543463048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 -- 172.21.15.39:0/2543463048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 msgr2=0x7feea8102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea8102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 -- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 msgr2=0x7feea81015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea81015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 -- 172.21.15.39:0/2543463048 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea81a6fa0 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feea6000700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea8101b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e9f0 tx=0x7fee9400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400c9c0 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea81a7290 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.006+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9400cb60 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.007+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94018680 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.007+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81a77d0 con 0x7feea806a940 2024-03-16T10:07:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.008+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea8068f90 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.010+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fee94010430 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.012+0000 7fee9f400700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 0x7fee8c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.012+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fee940cf0c0 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.012+0000 7feea6a00700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 0x7fee8c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.013+0000 7feea6a00700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 0x7fee8c0a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee98009990 tx=0x7fee98005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.019+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fee9409bc60 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.265+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feea802db70 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.265+0000 7fee9f400700 1 -- 172.21.15.39:0/2543463048 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee9409b560 con 0x7feea806a940 2024-03-16T10:07:42.853 INFO:teuthology.orchestra.run.smithi039.stdout:34359738461 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 msgr2=0x7fee8c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 0x7fee8c0a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee98009990 tx=0x7fee98005ab0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 msgr2=0x7feea8101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea8101b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee9400e9f0 tx=0x7fee9400ed00 comp rx=0 tx=0).stop 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 shutdown_connections 2024-03-16T10:07:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fee8c0a2920 0x7fee8c0a4de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea8108300 0x7feea8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feea8069f80 0x7feea81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.268+0000 7feead800700 1 --2- 172.21.15.39:0/2543463048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea806a940 0x7feea8101b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.269+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 >> 172.21.15.39:0/2543463048 conn(0x7feea8077f80 msgr2=0x7feea8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.269+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 shutdown_connections 2024-03-16T10:07:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:41.269+0000 7feead800700 1 -- 172.21.15.39:0/2543463048 wait complete. 2024-03-16T10:07:43.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.578+0000 7f3ddc000700 1 -- 172.21.15.39:0/3634750810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd4108320 msgr2=0x7f3dd4108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.578+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3634750810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd4108320 0x7f3dd4108700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009990 tx=0x7f3dc8009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 -- 172.21.15.39:0/3634750810 shutdown_connections 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3634750810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd406a920 0x7f3dd410f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3634750810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd4108320 0x7f3dd4108700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3634750810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4069f80 0x7f3dd406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 -- 172.21.15.39:0/3634750810 >> 172.21.15.39:0/3634750810 conn(0x7f3dd4078100 msgr2=0x7f3dd4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 -- 172.21.15.39:0/3634750810 shutdown_connections 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.579+0000 7f3ddc000700 1 -- 172.21.15.39:0/3634750810 wait complete. 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.580+0000 7f3ddc000700 1 Processor -- start 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.580+0000 7f3ddc000700 1 -- start start 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4069f80 0x7f3dd41a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 0x7f3dd419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd41125b0 con 0x7f3dd4069f80 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112430 con 0x7f3dd406a920 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3ddc000700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4112730 con 0x7f3dd4108320 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd8c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd9600700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4069f80 0x7f3dd41a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dda000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 0x7f3dd419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd8c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54570/0 (socket says 172.21.15.39:54570) 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd8c00700 1 -- 172.21.15.39:0/3482986634 learned_addr learned my addr 172.21.15.39:0/3482986634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd8c00700 1 -- 172.21.15.39:0/3482986634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 msgr2=0x7f3dd419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.581+0000 7f3dd8c00700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 0x7f3dd419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd8c00700 1 -- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4069f80 msgr2=0x7f3dd41a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd8c00700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4069f80 0x7f3dd41a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd8c00700 1 -- 172.21.15.39:0/3482986634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd419b190 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dda000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 0x7f3dd419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd8c00700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000e9a0 tx=0x7f3dc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0024070 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc000cbc0 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc001fc30 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.582+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd419b480 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.583+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41090a0 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.584+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd4068f90 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.586+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3dc0027070 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.588+0000 7f3dd2000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 0x7f3dbc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.588+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f3dc00d7140 con 0x7f3dd406a920 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.588+0000 7f3dd9600700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 0x7f3dbc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.589+0000 7f3dd9600700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 0x7f3dbc0a4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3dd419bb70 tx=0x7f3dc800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.591+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3dc00a3e40 con 0x7f3dd406a920 2024-03-16T10:07:43.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.871+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3dd4109360 con 0x7f3dd406a920 2024-03-16T10:07:43.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.872+0000 7f3dd2000700 1 -- 172.21.15.39:0/3482986634 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3dc00a3740 con 0x7f3dd406a920 2024-03-16T10:07:43.875 INFO:teuthology.orchestra.run.smithi039.stdout:146028888137 2024-03-16T10:07:43.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 msgr2=0x7f3dbc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 0x7f3dbc0a4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3dd419bb70 tx=0x7f3dc800b260 comp rx=0 tx=0).stop 2024-03-16T10:07:43.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 msgr2=0x7f3dd41a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000e9a0 tx=0x7f3dc000ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 shutdown_connections 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3dbc0a2910 0x7f3dbc0a4dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3dd4108320 0x7f3dd419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4069f80 0x7f3dd41a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 --2- 172.21.15.39:0/3482986634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dd406a920 0x7f3dd41a0db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.875+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 >> 172.21.15.39:0/3482986634 conn(0x7f3dd4078100 msgr2=0x7f3dd410d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.876+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 shutdown_connections 2024-03-16T10:07:43.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.876+0000 7f3ddc000700 1 -- 172.21.15.39:0/3482986634 wait complete. 2024-03-16T10:07:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 -- 172.21.15.39:0/1840643885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 msgr2=0x7fde3010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 --2- 172.21.15.39:0/1840643885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3010f160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde2800b0d0 tx=0x7fde2800b3e0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 -- 172.21.15.39:0/1840643885 shutdown_connections 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 --2- 172.21.15.39:0/1840643885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3010f160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 --2- 172.21.15.39:0/1840643885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde30108320 0x7fde30108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 --2- 172.21.15.39:0/1840643885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30069f80 0x7fde3006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 -- 172.21.15.39:0/1840643885 >> 172.21.15.39:0/1840643885 conn(0x7fde30078100 msgr2=0x7fde30078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 -- 172.21.15.39:0/1840643885 shutdown_connections 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.905+0000 7fde38400700 1 -- 172.21.15.39:0/1840643885 wait complete. 2024-03-16T10:07:43.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.906+0000 7fde38400700 1 Processor -- start 2024-03-16T10:07:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.907+0000 7fde38400700 1 -- start start 2024-03-16T10:07:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.907+0000 7fde38400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde30069f80 0x7fde3019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.907+0000 7fde38400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.907+0000 7fde38400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 0x7fde301a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde38400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde30118c50 con 0x7fde30069f80 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde38400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fde30118ad0 con 0x7fde30108320 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde38400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fde30118dd0 con 0x7fde3006a920 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:47948/0 (socket says 172.21.15.39:47948) 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 -- 172.21.15.39:0/4233323330 learned_addr learned my addr 172.21.15.39:0/4233323330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:43.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde36400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 0x7fde301a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 -- 172.21.15.39:0/4233323330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 msgr2=0x7fde301a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 0x7fde301a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 -- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde30069f80 msgr2=0x7fde3019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde30069f80 0x7fde3019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.908+0000 7fde35000700 1 -- 172.21.15.39:0/4233323330 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde301a4010 con 0x7fde3006a920 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde36400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 0x7fde301a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde35000700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e9a0 tx=0x7fde1c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00cbd0 con 0x7fde3006a920 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1c00cd70 con 0x7fde3006a920 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde301a4300 con 0x7fde3006a920 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0187c0 con 0x7fde3006a920 2024-03-16T10:07:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.909+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301abb80 con 0x7fde3006a920 2024-03-16T10:07:43.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.914+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde301039d0 con 0x7fde3006a920 2024-03-16T10:07:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.916+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fde1c01f020 con 0x7fde3006a920 2024-03-16T10:07:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.919+0000 7fde2e000700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 0x7fde180a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.919+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fde1c0cef70 con 0x7fde3006a920 2024-03-16T10:07:43.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.919+0000 7fde35a00700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 0x7fde180a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:43.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.920+0000 7fde35a00700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 0x7fde180a4e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde24009b60 tx=0x7fde24009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:43.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:43.921+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fde1c053740 con 0x7fde3006a920 2024-03-16T10:07:44.165 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:44 smithi039 ceph-mon[31510]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:44.165 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:44 smithi039 ceph-mon[31510]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-16T10:07:44.165 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:44 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3482986634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T10:07:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.163+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fde3002db70 con 0x7fde3006a920 2024-03-16T10:07:44.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.163+0000 7fde2e000700 1 -- 172.21.15.39:0/4233323330 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fde1c09bc70 con 0x7fde3006a920 2024-03-16T10:07:44.166 INFO:teuthology.orchestra.run.smithi039.stdout:55834574937 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 msgr2=0x7fde180a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 0x7fde180a4e90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fde24009b60 tx=0x7fde24009620 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 msgr2=0x7fde3019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00e9a0 tx=0x7fde1c00ed60 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 shutdown_connections 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fde180a29d0 0x7fde180a4e90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde30108320 0x7fde301a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde30069f80 0x7fde3019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 --2- 172.21.15.39:0/4233323330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde3006a920 0x7fde3019f540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:44.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 >> 172.21.15.39:0/4233323330 conn(0x7fde30078100 msgr2=0x7fde300ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:44.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 shutdown_connections 2024-03-16T10:07:44.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:44.167+0000 7fde38400700 1 -- 172.21.15.39:0/4233323330 wait complete. 2024-03-16T10:07:44.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:44 smithi105 ceph-mon[36559]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:44.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:44 smithi105 ceph-mon[36559]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-16T10:07:44.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:44 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3482986634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T10:07:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:44 smithi194 ceph-mon[36445]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:07:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:44 smithi194 ceph-mon[36445]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-03-16T10:07:44.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:44 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3482986634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T10:07:44.559 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:45.152 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574937 for osd.1 2024-03-16T10:07:45.152 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:45.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:45 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4233323330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T10:07:45.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724604 got 167503724610 for osd.5 2024-03-16T10:07:45.259 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:45.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:45 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4233323330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T10:07:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:45.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:45 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4233323330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T10:07:46.230 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:46 smithi039 ceph-mon[31510]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-16T10:07:46.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:46 smithi105 ceph-mon[36559]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-16T10:07:46.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:46 smithi194 ceph-mon[36445]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.266+0000 7f52b3c00700 1 -- 172.21.15.39:0/3995594666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac069f80 msgr2=0x7f52ac06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.266+0000 7f52b3c00700 1 --2- 172.21.15.39:0/3995594666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac069f80 0x7f52ac06a3e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5298009920 tx=0x7f5298009c30 comp rx=0 tx=0).stop 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 -- 172.21.15.39:0/3995594666 shutdown_connections 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 --2- 172.21.15.39:0/3995594666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac06a920 0x7f52ac10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 --2- 172.21.15.39:0/3995594666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac108320 0x7f52ac108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 --2- 172.21.15.39:0/3995594666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac069f80 0x7f52ac06a3e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 -- 172.21.15.39:0/3995594666 >> 172.21.15.39:0/3995594666 conn(0x7f52ac078100 msgr2=0x7f52ac078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 -- 172.21.15.39:0/3995594666 shutdown_connections 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.267+0000 7f52b3c00700 1 -- 172.21.15.39:0/3995594666 wait complete. 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.268+0000 7f52b3c00700 1 Processor -- start 2024-03-16T10:07:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 -- start start 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac069f80 0x7f52ac19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 0x7f52ac19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 0x7f52ac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1125d0 con 0x7f52ac069f80 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112450 con 0x7f52ac06a920 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b3c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112750 con 0x7f52ac108320 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52abe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 0x7f52ac19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.269+0000 7f52b1200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac069f80 0x7f52ac19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 0x7f52ac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52abe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 0x7f52ac19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54608/0 (socket says 172.21.15.39:54608) 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52abe00700 1 -- 172.21.15.39:0/4120685261 learned_addr learned my addr 172.21.15.39:0/4120685261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 -- 172.21.15.39:0/4120685261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 msgr2=0x7f52ac19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 0x7f52ac19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 -- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac069f80 msgr2=0x7f52ac19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac069f80 0x7f52ac19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 -- 172.21.15.39:0/4120685261 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac1990b0 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.270+0000 7f52b1c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 0x7f52ac198950 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007f10 tx=0x7f52a400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.271+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400dbd0 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.271+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a400dd70 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.271+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1993a0 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.271+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4015400 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.271+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1a5660 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.272+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac068f90 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.274+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f52a4005020 con 0x7f52ac108320 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.275+0000 7f52a9600700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 0x7f52940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.275+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f52a40ce680 con 0x7f52ac108320 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stdout:257698037810 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.275+0000 7f52b1200700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 0x7f52940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.276+0000 7f52b1200700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 0x7f52940a4dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52ac199a90 tx=0x7f52a0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.284+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f52a4010070 con 0x7f52ac108320 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.517+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f52ac02db70 con 0x7f52ac108320 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.518+0000 7f52a9600700 1 -- 172.21.15.39:0/4120685261 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f52a409b380 con 0x7f52ac108320 2024-03-16T10:07:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 msgr2=0x7f52940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 0x7f52940a4dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52ac199a90 tx=0x7f52a0009310 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 msgr2=0x7f52ac198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 0x7f52ac198950 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007f10 tx=0x7f52a400d3b0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 shutdown_connections 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f52940a2910 0x7f52940a4dd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f52ac108320 0x7f52ac198950 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52ac069f80 0x7f52ac19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 --2- 172.21.15.39:0/4120685261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52ac06a920 0x7f52ac19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 >> 172.21.15.39:0/4120685261 conn(0x7f52ac078100 msgr2=0x7f52ac0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 shutdown_connections 2024-03-16T10:07:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:46.521+0000 7f52b3c00700 1 -- 172.21.15.39:0/4120685261 wait complete. 2024-03-16T10:07:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:47 smithi194 ceph-mon[36445]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-16T10:07:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:47 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4120685261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T10:07:47.501 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:47 smithi039 ceph-mon[31510]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-16T10:07:47.501 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:47 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4120685261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T10:07:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:47 smithi105 ceph-mon[36559]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-16T10:07:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:47 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4120685261' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T10:07:47.693 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:47.694 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:47.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037804 got 257698037810 for osd.8 2024-03-16T10:07:47.725 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:48.150 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738461 for osd.0 2024-03-16T10:07:48.150 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:48.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888133 got 146028888137 for osd.4 2024-03-16T10:07:48.602 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:49.550 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:49 smithi039 ceph-mon[31510]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:49 smithi105 ceph-mon[36559]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:49 smithi194 ceph-mon[36445]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:51.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 -- 172.21.15.39:0/2460453007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce01071e0 msgr2=0x7f9ce006a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:51.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 --2- 172.21.15.39:0/2460453007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce01071e0 0x7f9ce006a970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009990 tx=0x7f9cd0009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 -- 172.21.15.39:0/2460453007 shutdown_connections 2024-03-16T10:07:51.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 --2- 172.21.15.39:0/2460453007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce0104d00 0x7f9ce0105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 --2- 172.21.15.39:0/2460453007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce01071e0 0x7f9ce006a970 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 --2- 172.21.15.39:0/2460453007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce006aeb0 0x7f9ce006b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 -- 172.21.15.39:0/2460453007 >> 172.21.15.39:0/2460453007 conn(0x7f9ce0078140 msgr2=0x7f9ce0078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.317+0000 7f9ce5400700 1 -- 172.21.15.39:0/2460453007 shutdown_connections 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.318+0000 7f9ce5400700 1 -- 172.21.15.39:0/2460453007 wait complete. 2024-03-16T10:07:51.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.319+0000 7f9ce5400700 1 Processor -- start 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.319+0000 7f9ce5400700 1 -- start start 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce006aeb0 0x7f9ce019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 0x7f9ce01a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0118b70 con 0x7f9ce01071e0 2024-03-16T10:07:51.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01189f0 con 0x7f9ce0104d00 2024-03-16T10:07:51.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9ce5400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0118cf0 con 0x7f9ce006aeb0 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9cdde00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9cde800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce006aeb0 0x7f9ce019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9cdde00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39728/0 (socket says 172.21.15.39:39728) 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9cdde00700 1 -- 172.21.15.39:0/1216208921 learned_addr learned my addr 172.21.15.39:0/1216208921 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.320+0000 7f9cdf200700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 0x7f9ce01a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 -- 172.21.15.39:0/1216208921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce006aeb0 msgr2=0x7f9ce019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce006aeb0 0x7f9ce019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 -- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 msgr2=0x7f9ce01a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 0x7f9ce01a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 -- 172.21.15.39:0/1216208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce01a4030 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdf200700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 0x7f9ce01a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cdde00700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e9a0 tx=0x7f9ccc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00cbd0 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc00cd70 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc018750 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce01a4320 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.321+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01abba0 con 0x7f9ce0104d00 2024-03-16T10:07:51.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.322+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce01017e0 con 0x7f9ce0104d00 2024-03-16T10:07:51.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.325+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9ccc010430 con 0x7f9ce0104d00 2024-03-16T10:07:51.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.327+0000 7f9cd7400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 0x7f9cc40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:51.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.327+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f9ccc0cf060 con 0x7f9ce0104d00 2024-03-16T10:07:51.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.327+0000 7f9cde800700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 0x7f9cc40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:51.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.328+0000 7f9cde800700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 0x7f9cc40a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006010 tx=0x7f9cd000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:51.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.331+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ccc09bdc0 con 0x7f9ce0104d00 2024-03-16T10:07:51.463 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:51 smithi039 ceph-mon[31510]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T10:07:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.564+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f9ce01abd70 con 0x7f9ce0104d00 2024-03-16T10:07:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.565+0000 7f9cd7400700 1 -- 172.21.15.39:0/1216208921 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9ccc09b6c0 con 0x7f9ce0104d00 2024-03-16T10:07:51.565 INFO:teuthology.orchestra.run.smithi039.stdout:292057776172 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 msgr2=0x7f9cc40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 0x7f9cc40a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0006010 tx=0x7f9cd000b260 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 msgr2=0x7f9ce019f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e9a0 tx=0x7f9ccc00ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 shutdown_connections 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.568+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9cc40a2890 0x7f9cc40a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce01071e0 0x7f9ce01a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce006aeb0 0x7f9ce019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 --2- 172.21.15.39:0/1216208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce0104d00 0x7f9ce019f560 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00e9a0 tx=0x7f9ccc00ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:51.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 >> 172.21.15.39:0/1216208921 conn(0x7f9ce0078140 msgr2=0x7f9ce0103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 shutdown_connections 2024-03-16T10:07:51.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:51.569+0000 7f9ce5400700 1 -- 172.21.15.39:0/1216208921 wait complete. 2024-03-16T10:07:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:51 smithi105 ceph-mon[36559]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T10:07:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:51 smithi194 ceph-mon[36445]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-03-16T10:07:52.572 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1216208921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T10:07:52.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1216208921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T10:07:52.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1216208921' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T10:07:53.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776167 got 292057776172 for osd.9 2024-03-16T10:07:53.546 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:07:54.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:53 smithi039 ceph-mon[31510]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:54.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:54.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:53 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:54.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:53 smithi194 ceph-mon[36445]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:54.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:54.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:53 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:54.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:53 smithi105 ceph-mon[36559]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-03-16T10:07:54.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:54.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:53 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:07:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:07:55.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:54 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:55.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:07:55.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:07:55.229 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:54 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 -- 172.21.15.39:0/1906672899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 msgr2=0x7f22941086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 --2- 172.21.15.39:0/1906672899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f22941086e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2284009990 tx=0x7f2284009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 -- 172.21.15.39:0/1906672899 shutdown_connections 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 --2- 172.21.15.39:0/1906672899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102ce0 0x7f229410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 --2- 172.21.15.39:0/1906672899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f22941086e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 --2- 172.21.15.39:0/1906672899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102320 0x7f22941027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 -- 172.21.15.39:0/1906672899 >> 172.21.15.39:0/1906672899 conn(0x7f2294079400 msgr2=0x7f2294079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 -- 172.21.15.39:0/1906672899 shutdown_connections 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.228+0000 7f229ac00700 1 -- 172.21.15.39:0/1906672899 wait complete. 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.229+0000 7f229ac00700 1 Processor -- start 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.229+0000 7f229ac00700 1 -- start start 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 0x7f229419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f2294198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22941125d0 con 0x7f2294102ce0 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2294112450 con 0x7f2294102320 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f229ac00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2294112750 con 0x7f2294108300 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39756/0 (socket says 172.21.15.39:39756) 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 -- 172.21.15.39:0/1985377393 learned_addr learned my addr 172.21.15.39:0/1985377393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2298c00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f2294198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293400700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 0x7f229419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 -- 172.21.15.39:0/1985377393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 msgr2=0x7f2294198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f2294198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 -- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 msgr2=0x7f229419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 0x7f229419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293e00700 1 -- 172.21.15.39:0/1985377393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2294199070 con 0x7f2294102320 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.230+0000 7f2293400700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 0x7f229419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f2293e00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2284005910 tx=0x7f22840047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284027070 con 0x7f2294102320 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228400b6c0 con 0x7f2294102320 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2294199300 con 0x7f2294102320 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284017650 con 0x7f2294102320 2024-03-16T10:07:55.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.231+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a55d0 con 0x7f2294102320 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.232+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2294068f90 con 0x7f2294102320 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.236+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f228402a070 con 0x7f2294102320 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.237+0000 7f2290c00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 0x7f22780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.237+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f22840cfcd0 con 0x7f2294102320 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.238+0000 7f2293400700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 0x7f22780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.239+0000 7f2293400700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 0x7f22780a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2294107210 tx=0x7f2280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:55.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.244+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f228405e740 con 0x7f2294102320 2024-03-16T10:07:55.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:07:55.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:07:55.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:54 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:07:55.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.485+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f229402db70 con 0x7f2294102320 2024-03-16T10:07:55.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.486+0000 7f2290c00700 1 -- 172.21.15.39:0/1985377393 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2284013070 con 0x7f2294102320 2024-03-16T10:07:55.487 INFO:teuthology.orchestra.run.smithi039.stdout:347892351013 2024-03-16T10:07:55.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 msgr2=0x7f22780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 0x7f22780a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2294107210 tx=0x7f2280009260 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 msgr2=0x7f229419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2284005910 tx=0x7f22840047b0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 shutdown_connections 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22780a2910 0x7f22780a4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2294108300 0x7f2294198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2294102320 0x7f229419e700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 --2- 172.21.15.39:0/1985377393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2294102ce0 0x7f229419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.489+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 >> 172.21.15.39:0/1985377393 conn(0x7f2294079400 msgr2=0x7f2294100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:55.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.490+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 shutdown_connections 2024-03-16T10:07:55.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.490+0000 7f229ac00700 1 -- 172.21.15.39:0/1985377393 wait complete. 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 -- 172.21.15.39:0/4001723270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760102320 msgr2=0x7ff7601027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 --2- 172.21.15.39:0/4001723270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760102320 0x7ff7601027a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009990 tx=0x7ff74c009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 -- 172.21.15.39:0/4001723270 shutdown_connections 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 --2- 172.21.15.39:0/4001723270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102ce0 0x7ff76010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 --2- 172.21.15.39:0/4001723270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760108300 0x7ff7601086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 --2- 172.21.15.39:0/4001723270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760102320 0x7ff7601027a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 -- 172.21.15.39:0/4001723270 >> 172.21.15.39:0/4001723270 conn(0x7ff760079400 msgr2=0x7ff760079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.634+0000 7ff767800700 1 -- 172.21.15.39:0/4001723270 shutdown_connections 2024-03-16T10:07:55.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.635+0000 7ff767800700 1 -- 172.21.15.39:0/4001723270 wait complete. 2024-03-16T10:07:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.636+0000 7ff767800700 1 Processor -- start 2024-03-16T10:07:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.636+0000 7ff767800700 1 -- start start 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 0x7ff76019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760102ce0 0x7ff76019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff760112580 con 0x7ff760102ce0 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff760112400 con 0x7ff760102320 2024-03-16T10:07:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff767800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff760112700 con 0x7ff760108300 2024-03-16T10:07:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff765800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff764e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 0x7ff76019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff765800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:48068/0 (socket says 172.21.15.39:48068) 2024-03-16T10:07:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff765800700 1 -- 172.21.15.39:0/1447461516 learned_addr learned my addr 172.21.15.39:0/1447461516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.637+0000 7ff75fe00700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760102ce0 0x7ff76019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff765800700 1 -- 172.21.15.39:0/1447461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 msgr2=0x7ff76019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff765800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 0x7ff76019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff765800700 1 -- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760102ce0 msgr2=0x7ff76019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff765800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760102ce0 0x7ff76019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff765800700 1 -- 172.21.15.39:0/1447461516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff760198fc0 con 0x7ff760108300 2024-03-16T10:07:55.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.638+0000 7ff764e00700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 0x7ff76019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff765800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff75800ef30 tx=0x7ff75800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580092e0 con 0x7ff760108300 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff758004bb0 con 0x7ff760108300 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004280 con 0x7ff760108300 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff760199250 con 0x7ff760108300 2024-03-16T10:07:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.639+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601a54c0 con 0x7ff760108300 2024-03-16T10:07:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.641+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7601039b0 con 0x7ff760108300 2024-03-16T10:07:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.642+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff75801b030 con 0x7ff760108300 2024-03-16T10:07:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.644+0000 7ff75d600700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 0x7ff7480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.644+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7ff7580ce850 con 0x7ff760108300 2024-03-16T10:07:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.644+0000 7ff764e00700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 0x7ff7480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:55.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.645+0000 7ff764e00700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 0x7ff7480a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff754011490 tx=0x7ff754009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:55.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.652+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff758022020 con 0x7ff760108300 2024-03-16T10:07:55.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.938+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff76002db70 con 0x7ff760108300 2024-03-16T10:07:55.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.939+0000 7ff75d600700 1 -- 172.21.15.39:0/1447461516 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff758010040 con 0x7ff760108300 2024-03-16T10:07:55.940 INFO:teuthology.orchestra.run.smithi039.stdout:313532612648 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 msgr2=0x7ff7480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 0x7ff7480a4dd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff754011490 tx=0x7ff754009260 comp rx=0 tx=0).stop 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 msgr2=0x7ff7601987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff75800ef30 tx=0x7ff75800c3a0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 shutdown_connections 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff7480a2910 0x7ff7480a4dd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff760108300 0x7ff7601987e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff760102320 0x7ff76019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.942+0000 7ff767800700 1 --2- 172.21.15.39:0/1447461516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff760102ce0 0x7ff76019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.943+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 >> 172.21.15.39:0/1447461516 conn(0x7ff760079400 msgr2=0x7ff760100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.943+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 shutdown_connections 2024-03-16T10:07:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:55.943+0000 7ff767800700 1 -- 172.21.15.39:0/1447461516 wait complete. 2024-03-16T10:07:56.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:55 smithi194 ceph-mon[36445]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-16T10:07:56.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1985377393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T10:07:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:55 smithi039 ceph-mon[31510]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-16T10:07:56.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1985377393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T10:07:56.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:55 smithi105 ceph-mon[36559]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-16T10:07:56.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1985377393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.799+0000 7f9d88e00700 1 -- 172.21.15.39:0/2937355386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 msgr2=0x7f9d8410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.799+0000 7f9d88e00700 1 --2- 172.21.15.39:0/2937355386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8410a970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009990 tx=0x7f9d74009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 -- 172.21.15.39:0/2937355386 shutdown_connections 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 --2- 172.21.15.39:0/2937355386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d84104ea0 0x7f9d84111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 --2- 172.21.15.39:0/2937355386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8410a970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 --2- 172.21.15.39:0/2937355386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d841044e0 0x7f9d84104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 -- 172.21.15.39:0/2937355386 >> 172.21.15.39:0/2937355386 conn(0x7f9d840fffb0 msgr2=0x7f9d841023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 -- 172.21.15.39:0/2937355386 shutdown_connections 2024-03-16T10:07:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.800+0000 7f9d88e00700 1 -- 172.21.15.39:0/2937355386 wait complete. 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.801+0000 7f9d88e00700 1 Processor -- start 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.801+0000 7f9d88e00700 1 -- start start 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.801+0000 7f9d88e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d88e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84104ea0 0x7f9d841a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d88e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d88e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84114820 con 0x7f9d84104ea0 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d88e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841146a0 con 0x7f9d841044e0 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d88e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841149a0 con 0x7f9d8410a590 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84104ea0 0x7f9d841a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39794/0 (socket says 172.21.15.39:39794) 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 -- 172.21.15.39:0/1500391528 learned_addr learned my addr 172.21.15.39:0/1500391528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d82800700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:57.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 -- 172.21.15.39:0/1500391528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 msgr2=0x7f9d8419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 -- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84104ea0 msgr2=0x7f9d841a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84104ea0 0x7f9d841a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d81e00700 1 -- 172.21.15.39:0/1500391528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8419b180 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.802+0000 7f9d82800700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d81e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005910 tx=0x7f9d740047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74027070 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7400b6c0 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8419b410 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74017650 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.803+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a76e0 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.804+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8410ca20 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.806+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9d7402a070 con 0x7f9d841044e0 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.808+0000 7f9d7aa00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 0x7f9d680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.808+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f9d740cfd50 con 0x7f9d841044e0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.808+0000 7f9d81400700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 0x7f9d680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.809+0000 7f9d81400700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 0x7f9d680a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d84109b30 tx=0x7f9d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:56.811+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9d7405e740 con 0x7f9d841044e0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.045+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9d8419bf00 con 0x7f9d841044e0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.045+0000 7f9d7aa00700 1 -- 172.21.15.39:0/1500391528 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d74013070 con 0x7f9d841044e0 2024-03-16T10:07:57.173 INFO:teuthology.orchestra.run.smithi039.stdout:111669149776 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 msgr2=0x7f9d680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 0x7f9d680a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d84109b30 tx=0x7f9d70009260 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 msgr2=0x7f9d841a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005910 tx=0x7f9d740047b0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 shutdown_connections 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9d680a2910 0x7f9d680a4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d8410a590 0x7f9d8419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d841044e0 0x7f9d841a0810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 --2- 172.21.15.39:0/1500391528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d84104ea0 0x7f9d841a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 >> 172.21.15.39:0/1500391528 conn(0x7f9d840fffb0 msgr2=0x7f9d84101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 shutdown_connections 2024-03-16T10:07:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:57.049+0000 7f9d88e00700 1 -- 172.21.15.39:0/1500391528 wait complete. 2024-03-16T10:07:57.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:56 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1447461516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T10:07:57.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:56 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1447461516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T10:07:57.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:56 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1447461516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T10:07:58.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:57 smithi039 ceph-mon[31510]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-16T10:07:58.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:57 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1500391528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T10:07:58.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612648 for osd.10 2024-03-16T10:07:58.365 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:57 smithi105 ceph-mon[36559]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-16T10:07:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:57 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1500391528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T10:07:58.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:57 smithi194 ceph-mon[36445]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-16T10:07:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:57 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1500391528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T10:07:58.719 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351007 got 347892351013 for osd.11 2024-03-16T10:07:58.719 DEBUG:teuthology.parallel:result is None 2024-03-16T10:07:58.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 -- 172.21.15.39:0/2616553677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e81088c0 msgr2=0x7fb6e8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:58.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2616553677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e81088c0 0x7fb6e8108d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009990 tx=0x7fb6d4009ca0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 -- 172.21.15.39:0/2616553677 shutdown_connections 2024-03-16T10:07:58.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2616553677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e8109280 0x7fb6e8115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2616553677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 0x7fb6e810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2616553677 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e81088c0 0x7fb6e8108d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 -- 172.21.15.39:0/2616553677 >> 172.21.15.39:0/2616553677 conn(0x7fb6e8075aa0 msgr2=0x7fb6e8077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 -- 172.21.15.39:0/2616553677 shutdown_connections 2024-03-16T10:07:58.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.949+0000 7fb6eca00700 1 -- 172.21.15.39:0/2616553677 wait complete. 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 Processor -- start 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 -- start start 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e8109280 0x7fb6e819f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 0x7fb6e81a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:58.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8118c40 con 0x7fb6e810e8c0 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8118ac0 con 0x7fb6e81088c0 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.951+0000 7fb6eca00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8118dc0 con 0x7fb6e8109280 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:39814/0 (socket says 172.21.15.39:39814) 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 -- 172.21.15.39:0/2131187220 learned_addr learned my addr 172.21.15.39:0/2131187220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e6800700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 0x7fb6e81a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5400700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e8109280 0x7fb6e819f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 -- 172.21.15.39:0/2131187220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e8109280 msgr2=0x7fb6e819f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e8109280 0x7fb6e819f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 -- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 msgr2=0x7fb6e81a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 0x7fb6e81a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 -- 172.21.15.39:0/2131187220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e81a4060 con 0x7fb6e81088c0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.952+0000 7fb6e5e00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800e9a0 tx=0x7fb6d800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.953+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800c9c0 con 0x7fb6e81088c0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.953+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e81a4350 con 0x7fb6e81088c0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.953+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d800cb60 con 0x7fb6e81088c0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.953+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8018680 con 0x7fb6e81088c0 2024-03-16T10:07:58.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.953+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e81abbd0 con 0x7fb6e81088c0 2024-03-16T10:07:58.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.954+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e8068f90 con 0x7fb6e81088c0 2024-03-16T10:07:58.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.956+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb6d800cd10 con 0x7fb6e81088c0 2024-03-16T10:07:58.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.958+0000 7fb6dea00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 0x7fb6cc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:07:58.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.958+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fb6d80ceec0 con 0x7fb6e81088c0 2024-03-16T10:07:58.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.959+0000 7fb6e5400700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 0x7fb6cc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:07:58.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.959+0000 7fb6e5400700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 0x7fb6cc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6e810de10 tx=0x7fb6d400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:07:58.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:58.961+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb6d809bbc0 con 0x7fb6e81088c0 2024-03-16T10:07:59.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.196+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb6e81abda0 con 0x7fb6e81088c0 2024-03-16T10:07:59.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.197+0000 7fb6dea00700 1 -- 172.21.15.39:0/2131187220 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6d809b4c0 con 0x7fb6e81088c0 2024-03-16T10:07:59.199 INFO:teuthology.orchestra.run.smithi039.stdout:77309411414 2024-03-16T10:07:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 msgr2=0x7fb6cc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 0x7fb6cc0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6e810de10 tx=0x7fb6d400b260 comp rx=0 tx=0).stop 2024-03-16T10:07:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 msgr2=0x7fb6e819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:07:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800e9a0 tx=0x7fb6d800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:07:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 shutdown_connections 2024-03-16T10:07:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb6cc0a2890 0x7fb6cc0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6e810e8c0 0x7fb6e81a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6e81088c0 0x7fb6e819f050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 --2- 172.21.15.39:0/2131187220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e8109280 0x7fb6e819f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:07:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.200+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 >> 172.21.15.39:0/2131187220 conn(0x7fb6e8075aa0 msgr2=0x7fb6e8075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:07:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.201+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 shutdown_connections 2024-03-16T10:07:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:07:59.201+0000 7fb6eca00700 1 -- 172.21.15.39:0/2131187220 wait complete. 2024-03-16T10:07:59.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149776 for osd.3 2024-03-16T10:07:59.463 DEBUG:teuthology.parallel:result is None 2024-03-16T10:08:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:59 smithi039 ceph-mon[31510]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-16T10:08:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:07:59 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2131187220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T10:08:00.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411414 for osd.2 2024-03-16T10:08:00.195 DEBUG:teuthology.parallel:result is None 2024-03-16T10:08:00.195 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-16T10:08:00.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T10:08:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:59 smithi105 ceph-mon[36559]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-16T10:08:00.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:07:59 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2131187220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T10:08:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:59 smithi194 ceph-mon[36445]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-03-16T10:08:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:07:59 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2131187220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T10:08:00.667 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:01 smithi039 ceph-mon[31510]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:01 smithi105 ceph-mon[36559]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:01 smithi194 ceph-mon[36445]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:02.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 -- 172.21.15.39:0/902690563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104ec0 msgr2=0x7f89cc111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:02.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 --2- 172.21.15.39:0/902690563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104ec0 0x7f89cc111380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b0d0 tx=0x7f89c400b3e0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 -- 172.21.15.39:0/902690563 shutdown_connections 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 --2- 172.21.15.39:0/902690563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104ec0 0x7f89cc111380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 --2- 172.21.15.39:0/902690563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89cc10a5b0 0x7f89cc10a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.959+0000 7f89d2c00700 1 --2- 172.21.15.39:0/902690563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104500 0x7f89cc104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.960+0000 7f89d2c00700 1 -- 172.21.15.39:0/902690563 >> 172.21.15.39:0/902690563 conn(0x7f89cc0fffd0 msgr2=0x7f89cc1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.960+0000 7f89d2c00700 1 -- 172.21.15.39:0/902690563 shutdown_connections 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.960+0000 7f89d2c00700 1 -- 172.21.15.39:0/902690563 wait complete. 2024-03-16T10:08:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.961+0000 7f89d2c00700 1 Processor -- start 2024-03-16T10:08:02.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.961+0000 7f89d2c00700 1 -- start start 2024-03-16T10:08:02.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104ec0 0x7f89cc1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89cc10a5b0 0x7f89cc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc114840 con 0x7f89cc10a5b0 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1146c0 con 0x7f89cc104500 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89d2c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1149c0 con 0x7f89cc104ec0 2024-03-16T10:08:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89cbe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89cbe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:38658/0 (socket says 172.21.15.39:38658) 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89cbe00700 1 -- 172.21.15.39:0/2548677190 learned_addr learned my addr 172.21.15.39:0/2548677190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.962+0000 7f89cb400700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104ec0 0x7f89cc1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 -- 172.21.15.39:0/2548677190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104ec0 msgr2=0x7f89cc1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104ec0 0x7f89cc1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 -- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89cc10a5b0 msgr2=0x7f89cc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89cc10a5b0 0x7f89cc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 -- 172.21.15.39:0/2548677190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc19b180 con 0x7f89cc104500 2024-03-16T10:08:02.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89cbe00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9a0 tx=0x7f89bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00cbd0 con 0x7f89cc104500 2024-03-16T10:08:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89bc00cd70 con 0x7f89cc104500 2024-03-16T10:08:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc018750 con 0x7f89cc104500 2024-03-16T10:08:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.963+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc19b470 con 0x7f89cc104500 2024-03-16T10:08:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.964+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc1a76e0 con 0x7f89cc104500 2024-03-16T10:08:02.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.965+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc068f90 con 0x7f89cc104500 2024-03-16T10:08:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.967+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f89bc010430 con 0x7f89cc104500 2024-03-16T10:08:02.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.968+0000 7f89c8c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 0x7f89b00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.968+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f89bc0cef80 con 0x7f89cc104500 2024-03-16T10:08:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.968+0000 7f89cb400700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 0x7f89b00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.969+0000 7f89cb400700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 0x7f89b00a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009680 tx=0x7f89b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:02.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:02.972+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f89bc09bc80 con 0x7f89cc104500 2024-03-16T10:08:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.195+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89cc062710 con 0x7f89b00a2910 2024-03-16T10:08:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.209+0000 7f89c8c00700 1 -- 172.21.15.39:0/2548677190 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340784 (secure 0 0 0) 0x7f89cc062710 con 0x7f89b00a2910 2024-03-16T10:08:03.218 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.218+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 msgr2=0x7f89b00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.218+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 0x7f89b00a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009680 tx=0x7f89b8009260 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.218+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 msgr2=0x7f89cc1a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.218+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00e9a0 tx=0x7f89bc00ecb0 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 shutdown_connections 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f89b00a2910 0x7f89b00a4dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89cc10a5b0 0x7f89cc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89cc104500 0x7f89cc1a0810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 --2- 172.21.15.39:0/2548677190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89cc104ec0 0x7f89cc1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 >> 172.21.15.39:0/2548677190 conn(0x7f89cc0fffd0 msgr2=0x7f89cc101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 shutdown_connections 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:03.219+0000 7f89d2c00700 1 -- 172.21.15.39:0/2548677190 wait complete. 2024-03-16T10:08:03.222 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-03-16T10:08:04.075 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-03-16T10:08:02.249928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660962,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":983,"num_read_kb":623,"num_write":433,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353224,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588600,"statfs":{"total":1151940427776,"available":1151578726400,"internally_reserved":0,"allocated":12689408,"data_stored":8004356,"data_compressed":362730,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":74,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005900"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216684+0000","last_change":"2024-03-16T10:06:04.221459+0000","last_active":"2024-03-16T10:07:42.216684+0000","last_peered":"2024-03-16T10:07:42.216684+0000","last_clean":"2024-03-16T10:07:42.216684+0000","last_became_active":"2024-03-16T10:06:04.221172+0000","last_became_peered":"2024-03-16T10:06:04.221172+0000","last_unstale":"2024-03-16T10:07:42.216684+0000","last_undegraded":"2024-03-16T10:07:42.216684+0000","last_fullsized":"2024-03-16T10:07:42.216684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:58:28.019985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213582+0000","last_change":"2024-03-16T10:06:04.205876+0000","last_active":"2024-03-16T10:07:42.213582+0000","last_peered":"2024-03-16T10:07:42.213582+0000","last_clean":"2024-03-16T10:07:42.213582+0000","last_became_active":"2024-03-16T10:06:04.205473+0000","last_became_peered":"2024-03-16T10:06:04.205473+0000","last_unstale":"2024-03-16T10:07:42.213582+0000","last_undegraded":"2024-03-16T10:07:42.213582+0000","last_fullsized":"2024-03-16T10:07:42.213582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:37.981148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217024+0000","last_change":"2024-03-16T10:06:04.197872+0000","last_active":"2024-03-16T10:07:42.217024+0000","last_peered":"2024-03-16T10:07:42.217024+0000","last_clean":"2024-03-16T10:07:42.217024+0000","last_became_active":"2024-03-16T10:06:04.197347+0000","last_became_peered":"2024-03-16T10:06:04.197347+0000","last_unstale":"2024-03-16T10:07:42.217024+0000","last_undegraded":"2024-03-16T10:07:42.217024+0000","last_fullsized":"2024-03-16T10:07:42.217024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:37.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190674+0000","last_change":"2024-03-16T10:06:04.196359+0000","last_active":"2024-03-16T10:07:42.190674+0000","last_peered":"2024-03-16T10:07:42.190674+0000","last_clean":"2024-03-16T10:07:42.190674+0000","last_became_active":"2024-03-16T10:06:04.196072+0000","last_became_peered":"2024-03-16T10:06:04.196072+0000","last_unstale":"2024-03-16T10:07:42.190674+0000","last_undegraded":"2024-03-16T10:07:42.190674+0000","last_fullsized":"2024-03-16T10:07:42.190674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:03.641536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196468+0000","last_change":"2024-03-16T10:07:14.198887+0000","last_active":"2024-03-16T10:07:42.196468+0000","last_peered":"2024-03-16T10:07:42.196468+0000","last_clean":"2024-03-16T10:07:42.196468+0000","last_became_active":"2024-03-16T10:07:14.198554+0000","last_became_peered":"2024-03-16T10:07:14.198554+0000","last_unstale":"2024-03-16T10:07:42.196468+0000","last_undegraded":"2024-03-16T10:07:42.196468+0000","last_fullsized":"2024-03-16T10:07:42.196468+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:25.665817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196080+0000","last_change":"2024-03-16T10:06:04.214055+0000","last_active":"2024-03-16T10:07:42.196080+0000","last_peered":"2024-03-16T10:07:42.196080+0000","last_clean":"2024-03-16T10:07:42.196080+0000","last_became_active":"2024-03-16T10:06:04.213792+0000","last_became_peered":"2024-03-16T10:06:04.213792+0000","last_unstale":"2024-03-16T10:07:42.196080+0000","last_undegraded":"2024-03-16T10:07:42.196080+0000","last_fullsized":"2024-03-16T10:07:42.196080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:50.142741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198356+0000","last_change":"2024-03-16T10:06:04.208874+0000","last_active":"2024-03-16T10:07:42.198356+0000","last_peered":"2024-03-16T10:07:42.198356+0000","last_clean":"2024-03-16T10:07:42.198356+0000","last_became_active":"2024-03-16T10:06:04.208553+0000","last_became_peered":"2024-03-16T10:06:04.208553+0000","last_unstale":"2024-03-16T10:07:42.198356+0000","last_undegraded":"2024-03-16T10:07:42.198356+0000","last_fullsized":"2024-03-16T10:07:42.198356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:33.885089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214693+0000","last_change":"2024-03-16T10:06:04.233487+0000","last_active":"2024-03-16T10:07:42.214693+0000","last_peered":"2024-03-16T10:07:42.214693+0000","last_clean":"2024-03-16T10:07:42.214693+0000","last_became_active":"2024-03-16T10:06:04.228530+0000","last_became_peered":"2024-03-16T10:06:04.228530+0000","last_unstale":"2024-03-16T10:07:42.214693+0000","last_undegraded":"2024-03-16T10:07:42.214693+0000","last_fullsized":"2024-03-16T10:07:42.214693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:53:12.270229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.209815+0000","last_change":"2024-03-16T10:06:04.190965+0000","last_active":"2024-03-16T10:07:42.209815+0000","last_peered":"2024-03-16T10:07:42.209815+0000","last_clean":"2024-03-16T10:07:42.209815+0000","last_became_active":"2024-03-16T10:06:04.190681+0000","last_became_peered":"2024-03-16T10:06:04.190681+0000","last_unstale":"2024-03-16T10:07:42.209815+0000","last_undegraded":"2024-03-16T10:07:42.209815+0000","last_fullsized":"2024-03-16T10:07:42.209815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:46.498680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196408+0000","last_change":"2024-03-16T10:06:04.198098+0000","last_active":"2024-03-16T10:07:42.196408+0000","last_peered":"2024-03-16T10:07:42.196408+0000","last_clean":"2024-03-16T10:07:42.196408+0000","last_became_active":"2024-03-16T10:06:04.197758+0000","last_became_peered":"2024-03-16T10:06:04.197758+0000","last_unstale":"2024-03-16T10:07:42.196408+0000","last_undegraded":"2024-03-16T10:07:42.196408+0000","last_fullsized":"2024-03-16T10:07:42.196408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:27.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211115+0000","last_change":"2024-03-16T10:06:04.195970+0000","last_active":"2024-03-16T10:07:42.211115+0000","last_peered":"2024-03-16T10:07:42.211115+0000","last_clean":"2024-03-16T10:07:42.211115+0000","last_became_active":"2024-03-16T10:06:04.195522+0000","last_became_peered":"2024-03-16T10:06:04.195522+0000","last_unstale":"2024-03-16T10:07:42.211115+0000","last_undegraded":"2024-03-16T10:07:42.211115+0000","last_fullsized":"2024-03-16T10:07:42.211115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:22.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190896+0000","last_change":"2024-03-16T10:06:04.196075+0000","last_active":"2024-03-16T10:07:42.190896+0000","last_peered":"2024-03-16T10:07:42.190896+0000","last_clean":"2024-03-16T10:07:42.190896+0000","last_became_active":"2024-03-16T10:06:04.195785+0000","last_became_peered":"2024-03-16T10:06:04.195785+0000","last_unstale":"2024-03-16T10:07:42.190896+0000","last_undegraded":"2024-03-16T10:07:42.190896+0000","last_fullsized":"2024-03-16T10:07:42.190896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:52:50.016787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196307+0000","last_change":"2024-03-16T10:06:04.206173+0000","last_active":"2024-03-16T10:07:42.196307+0000","last_peered":"2024-03-16T10:07:42.196307+0000","last_clean":"2024-03-16T10:07:42.196307+0000","last_became_active":"2024-03-16T10:06:04.205766+0000","last_became_peered":"2024-03-16T10:06:04.205766+0000","last_unstale":"2024-03-16T10:07:42.196307+0000","last_undegraded":"2024-03-16T10:07:42.196307+0000","last_fullsized":"2024-03-16T10:07:42.196307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:25:47.405044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213854+0000","last_change":"2024-03-16T10:06:04.196204+0000","last_active":"2024-03-16T10:07:42.213854+0000","last_peered":"2024-03-16T10:07:42.213854+0000","last_clean":"2024-03-16T10:07:42.213854+0000","last_became_active":"2024-03-16T10:06:04.194255+0000","last_became_peered":"2024-03-16T10:06:04.194255+0000","last_unstale":"2024-03-16T10:07:42.213854+0000","last_undegraded":"2024-03-16T10:07:42.213854+0000","last_fullsized":"2024-03-16T10:07:42.213854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:44.367876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214098+0000","last_change":"2024-03-16T10:06:04.203298+0000","last_active":"2024-03-16T10:07:42.214098+0000","last_peered":"2024-03-16T10:07:42.214098+0000","last_clean":"2024-03-16T10:07:42.214098+0000","last_became_active":"2024-03-16T10:06:04.202790+0000","last_became_peered":"2024-03-16T10:06:04.202790+0000","last_unstale":"2024-03-16T10:07:42.214098+0000","last_undegraded":"2024-03-16T10:07:42.214098+0000","last_fullsized":"2024-03-16T10:07:42.214098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:27.360861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217160+0000","last_change":"2024-03-16T10:06:04.191196+0000","last_active":"2024-03-16T10:07:42.217160+0000","last_peered":"2024-03-16T10:07:42.217160+0000","last_clean":"2024-03-16T10:07:42.217160+0000","last_became_active":"2024-03-16T10:06:04.190873+0000","last_became_peered":"2024-03-16T10:06:04.190873+0000","last_unstale":"2024-03-16T10:07:42.217160+0000","last_undegraded":"2024-03-16T10:07:42.217160+0000","last_fullsized":"2024-03-16T10:07:42.217160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:59.272388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216824+0000","last_change":"2024-03-16T10:06:04.221443+0000","last_active":"2024-03-16T10:07:42.216824+0000","last_peered":"2024-03-16T10:07:42.216824+0000","last_clean":"2024-03-16T10:07:42.216824+0000","last_became_active":"2024-03-16T10:06:04.221182+0000","last_became_peered":"2024-03-16T10:06:04.221182+0000","last_unstale":"2024-03-16T10:07:42.216824+0000","last_undegraded":"2024-03-16T10:07:42.216824+0000","last_fullsized":"2024-03-16T10:07:42.216824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:04:08.139784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198435+0000","last_change":"2024-03-16T10:06:04.208876+0000","last_active":"2024-03-16T10:07:42.198435+0000","last_peered":"2024-03-16T10:07:42.198435+0000","last_clean":"2024-03-16T10:07:42.198435+0000","last_became_active":"2024-03-16T10:06:04.208361+0000","last_became_peered":"2024-03-16T10:06:04.208361+0000","last_unstale":"2024-03-16T10:07:42.198435+0000","last_undegraded":"2024-03-16T10:07:42.198435+0000","last_fullsized":"2024-03-16T10:07:42.198435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:53.156832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211593+0000","last_change":"2024-03-16T10:06:04.228080+0000","last_active":"2024-03-16T10:07:42.211593+0000","last_peered":"2024-03-16T10:07:42.211593+0000","last_clean":"2024-03-16T10:07:42.211593+0000","last_became_active":"2024-03-16T10:06:04.227799+0000","last_became_peered":"2024-03-16T10:06:04.227799+0000","last_unstale":"2024-03-16T10:07:42.211593+0000","last_undegraded":"2024-03-16T10:07:42.211593+0000","last_fullsized":"2024-03-16T10:07:42.211593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:40:55.113354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927329+0000","last_change":"2024-03-16T10:06:08.349744+0000","last_active":"2024-03-16T10:07:13.927329+0000","last_peered":"2024-03-16T10:07:13.927329+0000","last_clean":"2024-03-16T10:07:13.927329+0000","last_became_active":"2024-03-16T10:06:08.349359+0000","last_became_peered":"2024-03-16T10:06:08.349359+0000","last_unstale":"2024-03-16T10:07:13.927329+0000","last_undegraded":"2024-03-16T10:07:13.927329+0000","last_fullsized":"2024-03-16T10:07:13.927329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:30:02.614189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":122,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:59.715655+0000","last_change":"2024-03-16T10:06:03.234836+0000","last_active":"2024-03-16T10:07:59.715655+0000","last_peered":"2024-03-16T10:07:59.715655+0000","last_clean":"2024-03-16T10:07:59.715655+0000","last_became_active":"2024-03-16T10:06:03.234436+0000","last_became_peered":"2024-03-16T10:06:03.234436+0000","last_unstale":"2024-03-16T10:07:59.715655+0000","last_undegraded":"2024-03-16T10:07:59.715655+0000","last_fullsized":"2024-03-16T10:07:59.715655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:52:34.069875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.206679+0000","last_change":"2024-03-16T10:06:04.222012+0000","last_active":"2024-03-16T10:07:42.206679+0000","last_peered":"2024-03-16T10:07:42.206679+0000","last_clean":"2024-03-16T10:07:42.206679+0000","last_became_active":"2024-03-16T10:06:04.221661+0000","last_became_peered":"2024-03-16T10:06:04.221661+0000","last_unstale":"2024-03-16T10:07:42.206679+0000","last_undegraded":"2024-03-16T10:07:42.206679+0000","last_fullsized":"2024-03-16T10:07:42.206679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:07.837752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.962913+0000","last_change":"2024-03-16T10:06:08.345658+0000","last_active":"2024-03-16T10:07:13.962913+0000","last_peered":"2024-03-16T10:07:13.962913+0000","last_clean":"2024-03-16T10:07:13.962913+0000","last_became_active":"2024-03-16T10:06:08.345291+0000","last_became_peered":"2024-03-16T10:06:08.345291+0000","last_unstale":"2024-03-16T10:07:13.962913+0000","last_undegraded":"2024-03-16T10:07:13.962913+0000","last_fullsized":"2024-03-16T10:07:13.962913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:43:39.734164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.064474+0000","last_change":"2024-03-16T10:06:03.244368+0000","last_active":"2024-03-16T10:07:14.064474+0000","last_peered":"2024-03-16T10:07:14.064474+0000","last_clean":"2024-03-16T10:07:14.064474+0000","last_became_active":"2024-03-16T10:06:03.243369+0000","last_became_peered":"2024-03-16T10:06:03.243369+0000","last_unstale":"2024-03-16T10:07:14.064474+0000","last_undegraded":"2024-03-16T10:07:14.064474+0000","last_fullsized":"2024-03-16T10:07:14.064474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:38.473988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217354+0000","last_change":"2024-03-16T10:06:04.217255+0000","last_active":"2024-03-16T10:07:42.217354+0000","last_peered":"2024-03-16T10:07:42.217354+0000","last_clean":"2024-03-16T10:07:42.217354+0000","last_became_active":"2024-03-16T10:06:04.216659+0000","last_became_peered":"2024-03-16T10:06:04.216659+0000","last_unstale":"2024-03-16T10:07:42.217354+0000","last_undegraded":"2024-03-16T10:07:42.217354+0000","last_fullsized":"2024-03-16T10:07:42.217354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:06.081507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963349+0000","last_change":"2024-03-16T10:06:08.346622+0000","last_active":"2024-03-16T10:07:13.963349+0000","last_peered":"2024-03-16T10:07:13.963349+0000","last_clean":"2024-03-16T10:07:13.963349+0000","last_became_active":"2024-03-16T10:06:08.346209+0000","last_became_peered":"2024-03-16T10:06:08.346209+0000","last_unstale":"2024-03-16T10:07:13.963349+0000","last_undegraded":"2024-03-16T10:07:13.963349+0000","last_fullsized":"2024-03-16T10:07:13.963349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:46:07.305218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924502+0000","last_change":"2024-03-16T10:06:03.227303+0000","last_active":"2024-03-16T10:07:13.924502+0000","last_peered":"2024-03-16T10:07:13.924502+0000","last_clean":"2024-03-16T10:07:13.924502+0000","last_became_active":"2024-03-16T10:06:03.226953+0000","last_became_peered":"2024-03-16T10:06:03.226953+0000","last_unstale":"2024-03-16T10:07:13.924502+0000","last_undegraded":"2024-03-16T10:07:13.924502+0000","last_fullsized":"2024-03-16T10:07:13.924502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:28:40.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211505+0000","last_change":"2024-03-16T10:06:04.220502+0000","last_active":"2024-03-16T10:07:42.211505+0000","last_peered":"2024-03-16T10:07:42.211505+0000","last_clean":"2024-03-16T10:07:42.211505+0000","last_became_active":"2024-03-16T10:06:04.220057+0000","last_became_peered":"2024-03-16T10:06:04.220057+0000","last_unstale":"2024-03-16T10:07:42.211505+0000","last_undegraded":"2024-03-16T10:07:42.211505+0000","last_fullsized":"2024-03-16T10:07:42.211505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:46:30.972382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963055+0000","last_change":"2024-03-16T10:06:08.346548+0000","last_active":"2024-03-16T10:07:13.963055+0000","last_peered":"2024-03-16T10:07:13.963055+0000","last_clean":"2024-03-16T10:07:13.963055+0000","last_became_active":"2024-03-16T10:06:08.346145+0000","last_became_peered":"2024-03-16T10:06:08.346145+0000","last_unstale":"2024-03-16T10:07:13.963055+0000","last_undegraded":"2024-03-16T10:07:13.963055+0000","last_fullsized":"2024-03-16T10:07:13.963055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:49.221773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924325+0000","last_change":"2024-03-16T10:06:03.222623+0000","last_active":"2024-03-16T10:07:13.924325+0000","last_peered":"2024-03-16T10:07:13.924325+0000","last_clean":"2024-03-16T10:07:13.924325+0000","last_became_active":"2024-03-16T10:06:03.222300+0000","last_became_peered":"2024-03-16T10:06:03.222300+0000","last_unstale":"2024-03-16T10:07:13.924325+0000","last_undegraded":"2024-03-16T10:07:13.924325+0000","last_fullsized":"2024-03-16T10:07:13.924325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:49.397327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.200074+0000","last_change":"2024-03-16T10:07:14.202077+0000","last_active":"2024-03-16T10:07:42.200074+0000","last_peered":"2024-03-16T10:07:42.200074+0000","last_clean":"2024-03-16T10:07:42.200074+0000","last_became_active":"2024-03-16T10:07:14.201647+0000","last_became_peered":"2024-03-16T10:07:14.201647+0000","last_unstale":"2024-03-16T10:07:42.200074+0000","last_undegraded":"2024-03-16T10:07:42.200074+0000","last_fullsized":"2024-03-16T10:07:42.200074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:30.698520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926663+0000","last_change":"2024-03-16T10:06:08.330170+0000","last_active":"2024-03-16T10:07:13.926663+0000","last_peered":"2024-03-16T10:07:13.926663+0000","last_clean":"2024-03-16T10:07:13.926663+0000","last_became_active":"2024-03-16T10:06:08.329699+0000","last_became_peered":"2024-03-16T10:06:08.329699+0000","last_unstale":"2024-03-16T10:07:13.926663+0000","last_undegraded":"2024-03-16T10:07:13.926663+0000","last_fullsized":"2024-03-16T10:07:13.926663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:50.571845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":111,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:55.674552+0000","last_change":"2024-03-16T10:06:03.228039+0000","last_active":"2024-03-16T10:07:55.674552+0000","last_peered":"2024-03-16T10:07:55.674552+0000","last_clean":"2024-03-16T10:07:55.674552+0000","last_became_active":"2024-03-16T10:06:03.227715+0000","last_became_peered":"2024-03-16T10:06:03.227715+0000","last_unstale":"2024-03-16T10:07:55.674552+0000","last_undegraded":"2024-03-16T10:07:55.674552+0000","last_fullsized":"2024-03-16T10:07:55.674552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:13:08.980781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.188434+0000","last_change":"2024-03-16T10:06:04.214274+0000","last_active":"2024-03-16T10:07:42.188434+0000","last_peered":"2024-03-16T10:07:42.188434+0000","last_clean":"2024-03-16T10:07:42.188434+0000","last_became_active":"2024-03-16T10:06:04.213979+0000","last_became_peered":"2024-03-16T10:06:04.213979+0000","last_unstale":"2024-03-16T10:07:42.188434+0000","last_undegraded":"2024-03-16T10:07:42.188434+0000","last_fullsized":"2024-03-16T10:07:42.188434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:12.684609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927412+0000","last_change":"2024-03-16T10:06:08.346874+0000","last_active":"2024-03-16T10:07:13.927412+0000","last_peered":"2024-03-16T10:07:13.927412+0000","last_clean":"2024-03-16T10:07:13.927412+0000","last_became_active":"2024-03-16T10:06:08.346450+0000","last_became_peered":"2024-03-16T10:06:08.346450+0000","last_unstale":"2024-03-16T10:07:13.927412+0000","last_undegraded":"2024-03-16T10:07:13.927412+0000","last_fullsized":"2024-03-16T10:07:13.927412+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:09:12.159483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196008+0000","last_change":"2024-03-16T10:06:03.236088+0000","last_active":"2024-03-16T10:07:14.196008+0000","last_peered":"2024-03-16T10:07:14.196008+0000","last_clean":"2024-03-16T10:07:14.196008+0000","last_became_active":"2024-03-16T10:06:03.235768+0000","last_became_peered":"2024-03-16T10:06:03.235768+0000","last_unstale":"2024-03-16T10:07:14.196008+0000","last_undegraded":"2024-03-16T10:07:14.196008+0000","last_fullsized":"2024-03-16T10:07:14.196008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:57:38.813913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.207966+0000","last_change":"2024-03-16T10:06:04.224331+0000","last_active":"2024-03-16T10:07:42.207966+0000","last_peered":"2024-03-16T10:07:42.207966+0000","last_clean":"2024-03-16T10:07:42.207966+0000","last_became_active":"2024-03-16T10:06:04.223946+0000","last_became_peered":"2024-03-16T10:06:04.223946+0000","last_unstale":"2024-03-16T10:07:42.207966+0000","last_undegraded":"2024-03-16T10:07:42.207966+0000","last_fullsized":"2024-03-16T10:07:42.207966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:58:56.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927251+0000","last_change":"2024-03-16T10:06:08.344757+0000","last_active":"2024-03-16T10:07:13.927251+0000","last_peered":"2024-03-16T10:07:13.927251+0000","last_clean":"2024-03-16T10:07:13.927251+0000","last_became_active":"2024-03-16T10:06:08.344275+0000","last_became_peered":"2024-03-16T10:06:08.344275+0000","last_unstale":"2024-03-16T10:07:13.927251+0000","last_undegraded":"2024-03-16T10:07:13.927251+0000","last_fullsized":"2024-03-16T10:07:13.927251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:31:18.545734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.197197+0000","last_change":"2024-03-16T10:06:03.239619+0000","last_active":"2024-03-16T10:07:14.197197+0000","last_peered":"2024-03-16T10:07:14.197197+0000","last_clean":"2024-03-16T10:07:14.197197+0000","last_became_active":"2024-03-16T10:06:03.238982+0000","last_became_peered":"2024-03-16T10:06:03.238982+0000","last_unstale":"2024-03-16T10:07:14.197197+0000","last_undegraded":"2024-03-16T10:07:14.197197+0000","last_fullsized":"2024-03-16T10:07:14.197197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:20.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213705+0000","last_change":"2024-03-16T10:06:04.221897+0000","last_active":"2024-03-16T10:07:42.213705+0000","last_peered":"2024-03-16T10:07:42.213705+0000","last_clean":"2024-03-16T10:07:42.213705+0000","last_became_active":"2024-03-16T10:06:04.218243+0000","last_became_peered":"2024-03-16T10:06:04.218243+0000","last_unstale":"2024-03-16T10:07:42.213705+0000","last_undegraded":"2024-03-16T10:07:42.213705+0000","last_fullsized":"2024-03-16T10:07:42.213705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:25.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.933682+0000","last_change":"2024-03-16T10:06:08.352722+0000","last_active":"2024-03-16T10:07:13.933682+0000","last_peered":"2024-03-16T10:07:13.933682+0000","last_clean":"2024-03-16T10:07:13.933682+0000","last_became_active":"2024-03-16T10:06:08.352292+0000","last_became_peered":"2024-03-16T10:06:08.352292+0000","last_unstale":"2024-03-16T10:07:13.933682+0000","last_undegraded":"2024-03-16T10:07:13.933682+0000","last_fullsized":"2024-03-16T10:07:13.933682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:56:23.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195477+0000","last_change":"2024-03-16T10:06:03.224143+0000","last_active":"2024-03-16T10:07:14.195477+0000","last_peered":"2024-03-16T10:07:14.195477+0000","last_clean":"2024-03-16T10:07:14.195477+0000","last_became_active":"2024-03-16T10:06:03.223810+0000","last_became_peered":"2024-03-16T10:06:03.223810+0000","last_unstale":"2024-03-16T10:07:14.195477+0000","last_undegraded":"2024-03-16T10:07:14.195477+0000","last_fullsized":"2024-03-16T10:07:14.195477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:26.724292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212935+0000","last_change":"2024-03-16T10:06:04.232758+0000","last_active":"2024-03-16T10:07:42.212935+0000","last_peered":"2024-03-16T10:07:42.212935+0000","last_clean":"2024-03-16T10:07:42.212935+0000","last_became_active":"2024-03-16T10:06:04.232371+0000","last_became_peered":"2024-03-16T10:06:04.232371+0000","last_unstale":"2024-03-16T10:07:42.212935+0000","last_undegraded":"2024-03-16T10:07:42.212935+0000","last_fullsized":"2024-03-16T10:07:42.212935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:00.527708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.063109+0000","last_change":"2024-03-16T10:06:08.345360+0000","last_active":"2024-03-16T10:07:14.063109+0000","last_peered":"2024-03-16T10:07:14.063109+0000","last_clean":"2024-03-16T10:07:14.063109+0000","last_became_active":"2024-03-16T10:06:08.336245+0000","last_became_peered":"2024-03-16T10:06:08.336245+0000","last_unstale":"2024-03-16T10:07:14.063109+0000","last_undegraded":"2024-03-16T10:07:14.063109+0000","last_fullsized":"2024-03-16T10:07:14.063109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:14:33.324555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":195,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.723288+0000","last_change":"2024-03-16T10:06:03.237015+0000","last_active":"2024-03-16T10:08:00.723288+0000","last_peered":"2024-03-16T10:08:00.723288+0000","last_clean":"2024-03-16T10:08:00.723288+0000","last_became_active":"2024-03-16T10:06:03.236722+0000","last_became_peered":"2024-03-16T10:06:03.236722+0000","last_unstale":"2024-03-16T10:08:00.723288+0000","last_undegraded":"2024-03-16T10:08:00.723288+0000","last_fullsized":"2024-03-16T10:08:00.723288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:10:03.403854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210293+0000","last_change":"2024-03-16T10:06:04.209017+0000","last_active":"2024-03-16T10:07:42.210293+0000","last_peered":"2024-03-16T10:07:42.210293+0000","last_clean":"2024-03-16T10:07:42.210293+0000","last_became_active":"2024-03-16T10:06:04.208665+0000","last_became_peered":"2024-03-16T10:06:04.208665+0000","last_unstale":"2024-03-16T10:07:42.210293+0000","last_undegraded":"2024-03-16T10:07:42.210293+0000","last_fullsized":"2024-03-16T10:07:42.210293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:27.074480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928969+0000","last_change":"2024-03-16T10:06:08.340510+0000","last_active":"2024-03-16T10:07:13.928969+0000","last_peered":"2024-03-16T10:07:13.928969+0000","last_clean":"2024-03-16T10:07:13.928969+0000","last_became_active":"2024-03-16T10:06:08.340070+0000","last_became_peered":"2024-03-16T10:06:08.340070+0000","last_unstale":"2024-03-16T10:07:13.928969+0000","last_undegraded":"2024-03-16T10:07:13.928969+0000","last_fullsized":"2024-03-16T10:07:13.928969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:33:55.038855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926601+0000","last_change":"2024-03-16T10:06:03.225770+0000","last_active":"2024-03-16T10:07:13.926601+0000","last_peered":"2024-03-16T10:07:13.926601+0000","last_clean":"2024-03-16T10:07:13.926601+0000","last_became_active":"2024-03-16T10:06:03.223734+0000","last_became_peered":"2024-03-16T10:06:03.223734+0000","last_unstale":"2024-03-16T10:07:13.926601+0000","last_undegraded":"2024-03-16T10:07:13.926601+0000","last_fullsized":"2024-03-16T10:07:13.926601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:13.764147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212151+0000","last_change":"2024-03-16T10:07:14.197803+0000","last_active":"2024-03-16T10:07:42.212151+0000","last_peered":"2024-03-16T10:07:42.212151+0000","last_clean":"2024-03-16T10:07:42.212151+0000","last_became_active":"2024-03-16T10:07:14.197195+0000","last_became_peered":"2024-03-16T10:07:14.197195+0000","last_unstale":"2024-03-16T10:07:42.212151+0000","last_undegraded":"2024-03-16T10:07:42.212151+0000","last_fullsized":"2024-03-16T10:07:42.212151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:01:03.480157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963468+0000","last_change":"2024-03-16T10:06:08.345595+0000","last_active":"2024-03-16T10:07:13.963468+0000","last_peered":"2024-03-16T10:07:13.963468+0000","last_clean":"2024-03-16T10:07:13.963468+0000","last_became_active":"2024-03-16T10:06:08.345205+0000","last_became_peered":"2024-03-16T10:06:08.345205+0000","last_unstale":"2024-03-16T10:07:13.963468+0000","last_undegraded":"2024-03-16T10:07:13.963468+0000","last_fullsized":"2024-03-16T10:07:13.963468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:37:51.067056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195692+0000","last_change":"2024-03-16T10:06:03.219716+0000","last_active":"2024-03-16T10:07:14.195692+0000","last_peered":"2024-03-16T10:07:14.195692+0000","last_clean":"2024-03-16T10:07:14.195692+0000","last_became_active":"2024-03-16T10:06:03.219375+0000","last_became_peered":"2024-03-16T10:06:03.219375+0000","last_unstale":"2024-03-16T10:07:14.195692+0000","last_undegraded":"2024-03-16T10:07:14.195692+0000","last_fullsized":"2024-03-16T10:07:14.195692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:23:51.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211478+0000","last_change":"2024-03-16T10:06:04.193605+0000","last_active":"2024-03-16T10:07:42.211478+0000","last_peered":"2024-03-16T10:07:42.211478+0000","last_clean":"2024-03-16T10:07:42.211478+0000","last_became_active":"2024-03-16T10:06:04.193295+0000","last_became_peered":"2024-03-16T10:06:04.193295+0000","last_unstale":"2024-03-16T10:07:42.211478+0000","last_undegraded":"2024-03-16T10:07:42.211478+0000","last_fullsized":"2024-03-16T10:07:42.211478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:10:04.140279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.931040+0000","last_change":"2024-03-16T10:06:08.338862+0000","last_active":"2024-03-16T10:07:13.931040+0000","last_peered":"2024-03-16T10:07:13.931040+0000","last_clean":"2024-03-16T10:07:13.931040+0000","last_became_active":"2024-03-16T10:06:08.337221+0000","last_became_peered":"2024-03-16T10:06:08.337221+0000","last_unstale":"2024-03-16T10:07:13.931040+0000","last_undegraded":"2024-03-16T10:07:13.931040+0000","last_fullsized":"2024-03-16T10:07:13.931040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:28:51.251299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":235,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:01.683265+0000","last_change":"2024-03-16T10:06:03.220555+0000","last_active":"2024-03-16T10:08:01.683265+0000","last_peered":"2024-03-16T10:08:01.683265+0000","last_clean":"2024-03-16T10:08:01.683265+0000","last_became_active":"2024-03-16T10:06:03.220224+0000","last_became_peered":"2024-03-16T10:06:03.220224+0000","last_unstale":"2024-03-16T10:08:01.683265+0000","last_undegraded":"2024-03-16T10:08:01.683265+0000","last_fullsized":"2024-03-16T10:08:01.683265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:12:47.568132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":171,"num_read_kb":147,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.191212+0000","last_change":"2024-03-16T10:06:04.195957+0000","last_active":"2024-03-16T10:07:42.191212+0000","last_peered":"2024-03-16T10:07:42.191212+0000","last_clean":"2024-03-16T10:07:42.191212+0000","last_became_active":"2024-03-16T10:06:04.195539+0000","last_became_peered":"2024-03-16T10:06:04.195539+0000","last_unstale":"2024-03-16T10:07:42.191212+0000","last_undegraded":"2024-03-16T10:07:42.191212+0000","last_fullsized":"2024-03-16T10:07:42.191212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:39:45.035936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196250+0000","last_change":"2024-03-16T10:06:08.347941+0000","last_active":"2024-03-16T10:07:14.196250+0000","last_peered":"2024-03-16T10:07:14.196250+0000","last_clean":"2024-03-16T10:07:14.196250+0000","last_became_active":"2024-03-16T10:06:08.347517+0000","last_became_peered":"2024-03-16T10:06:08.347517+0000","last_unstale":"2024-03-16T10:07:14.196250+0000","last_undegraded":"2024-03-16T10:07:14.196250+0000","last_fullsized":"2024-03-16T10:07:14.196250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:51:08.696206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927465+0000","last_change":"2024-03-16T10:06:03.235652+0000","last_active":"2024-03-16T10:07:13.927465+0000","last_peered":"2024-03-16T10:07:13.927465+0000","last_clean":"2024-03-16T10:07:13.927465+0000","last_became_active":"2024-03-16T10:06:03.235219+0000","last_became_peered":"2024-03-16T10:06:03.235219+0000","last_unstale":"2024-03-16T10:07:13.927465+0000","last_undegraded":"2024-03-16T10:07:13.927465+0000","last_fullsized":"2024-03-16T10:07:13.927465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:23.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214350+0000","last_change":"2024-03-16T10:06:04.225460+0000","last_active":"2024-03-16T10:07:42.214350+0000","last_peered":"2024-03-16T10:07:42.214350+0000","last_clean":"2024-03-16T10:07:42.214350+0000","last_became_active":"2024-03-16T10:06:04.225019+0000","last_became_peered":"2024-03-16T10:06:04.225019+0000","last_unstale":"2024-03-16T10:07:42.214350+0000","last_undegraded":"2024-03-16T10:07:42.214350+0000","last_fullsized":"2024-03-16T10:07:42.214350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:42:06.525298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.925840+0000","last_change":"2024-03-16T10:06:08.344775+0000","last_active":"2024-03-16T10:07:13.925840+0000","last_peered":"2024-03-16T10:07:13.925840+0000","last_clean":"2024-03-16T10:07:13.925840+0000","last_became_active":"2024-03-16T10:06:08.344360+0000","last_became_peered":"2024-03-16T10:06:08.344360+0000","last_unstale":"2024-03-16T10:07:13.925840+0000","last_undegraded":"2024-03-16T10:07:13.925840+0000","last_fullsized":"2024-03-16T10:07:13.925840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:26.710955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195992+0000","last_change":"2024-03-16T10:06:03.236887+0000","last_active":"2024-03-16T10:07:14.195992+0000","last_peered":"2024-03-16T10:07:14.195992+0000","last_clean":"2024-03-16T10:07:14.195992+0000","last_became_active":"2024-03-16T10:06:03.236454+0000","last_became_peered":"2024-03-16T10:06:03.236454+0000","last_unstale":"2024-03-16T10:07:14.195992+0000","last_undegraded":"2024-03-16T10:07:14.195992+0000","last_fullsized":"2024-03-16T10:07:14.195992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:03.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.203957+0000","last_change":"2024-03-16T10:06:04.220008+0000","last_active":"2024-03-16T10:07:42.203957+0000","last_peered":"2024-03-16T10:07:42.203957+0000","last_clean":"2024-03-16T10:07:42.203957+0000","last_became_active":"2024-03-16T10:06:04.219683+0000","last_became_peered":"2024-03-16T10:06:04.219683+0000","last_unstale":"2024-03-16T10:07:42.203957+0000","last_undegraded":"2024-03-16T10:07:42.203957+0000","last_fullsized":"2024-03-16T10:07:42.203957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:35.577474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.930260+0000","last_change":"2024-03-16T10:06:08.352793+0000","last_active":"2024-03-16T10:07:13.930260+0000","last_peered":"2024-03-16T10:07:13.930260+0000","last_clean":"2024-03-16T10:07:13.930260+0000","last_became_active":"2024-03-16T10:06:08.352272+0000","last_became_peered":"2024-03-16T10:06:08.352272+0000","last_unstale":"2024-03-16T10:07:13.930260+0000","last_undegraded":"2024-03-16T10:07:13.930260+0000","last_fullsized":"2024-03-16T10:07:13.930260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:39.835078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:57.747728+0000","last_change":"2024-03-16T10:06:03.224301+0000","last_active":"2024-03-16T10:07:57.747728+0000","last_peered":"2024-03-16T10:07:57.747728+0000","last_clean":"2024-03-16T10:07:57.747728+0000","last_became_active":"2024-03-16T10:06:03.223899+0000","last_became_peered":"2024-03-16T10:06:03.223899+0000","last_unstale":"2024-03-16T10:07:57.747728+0000","last_undegraded":"2024-03-16T10:07:57.747728+0000","last_fullsized":"2024-03-16T10:07:57.747728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:39.244106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198542+0000","last_change":"2024-03-16T10:06:04.193649+0000","last_active":"2024-03-16T10:07:42.198542+0000","last_peered":"2024-03-16T10:07:42.198542+0000","last_clean":"2024-03-16T10:07:42.198542+0000","last_became_active":"2024-03-16T10:06:04.193275+0000","last_became_peered":"2024-03-16T10:06:04.193275+0000","last_unstale":"2024-03-16T10:07:42.198542+0000","last_undegraded":"2024-03-16T10:07:42.198542+0000","last_fullsized":"2024-03-16T10:07:42.198542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:59.597769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198270+0000","last_change":"2024-03-16T10:06:08.349918+0000","last_active":"2024-03-16T10:07:14.198270+0000","last_peered":"2024-03-16T10:07:14.198270+0000","last_clean":"2024-03-16T10:07:14.198270+0000","last_became_active":"2024-03-16T10:06:08.349506+0000","last_became_peered":"2024-03-16T10:06:08.349506+0000","last_unstale":"2024-03-16T10:07:14.198270+0000","last_undegraded":"2024-03-16T10:07:14.198270+0000","last_fullsized":"2024-03-16T10:07:14.198270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:08.146594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.946883+0000","last_change":"2024-03-16T10:07:13.946883+0000","last_active":"2024-03-16T10:07:13.946883+0000","last_peered":"2024-03-16T10:07:13.946883+0000","last_clean":"2024-03-16T10:07:13.946883+0000","last_became_active":"2024-03-16T10:07:13.934745+0000","last_became_peered":"2024-03-16T10:07:13.934745+0000","last_unstale":"2024-03-16T10:07:13.946883+0000","last_undegraded":"2024-03-16T10:07:13.946883+0000","last_fullsized":"2024-03-16T10:07:13.946883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:26:42.936333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.201757+0000","last_change":"2024-03-16T10:06:04.194629+0000","last_active":"2024-03-16T10:07:42.201757+0000","last_peered":"2024-03-16T10:07:42.201757+0000","last_clean":"2024-03-16T10:07:42.201757+0000","last_became_active":"2024-03-16T10:06:04.194306+0000","last_became_peered":"2024-03-16T10:06:04.194306+0000","last_unstale":"2024-03-16T10:07:42.201757+0000","last_undegraded":"2024-03-16T10:07:42.201757+0000","last_fullsized":"2024-03-16T10:07:42.201757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:36.455569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195547+0000","last_change":"2024-03-16T10:06:08.349269+0000","last_active":"2024-03-16T10:07:14.195547+0000","last_peered":"2024-03-16T10:07:14.195547+0000","last_clean":"2024-03-16T10:07:14.195547+0000","last_became_active":"2024-03-16T10:06:08.348824+0000","last_became_peered":"2024-03-16T10:06:08.348824+0000","last_unstale":"2024-03-16T10:07:14.195547+0000","last_undegraded":"2024-03-16T10:07:14.195547+0000","last_fullsized":"2024-03-16T10:07:14.195547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:26.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924724+0000","last_change":"2024-03-16T10:06:03.221066+0000","last_active":"2024-03-16T10:07:13.924724+0000","last_peered":"2024-03-16T10:07:13.924724+0000","last_clean":"2024-03-16T10:07:13.924724+0000","last_became_active":"2024-03-16T10:06:03.219908+0000","last_became_peered":"2024-03-16T10:06:03.219908+0000","last_unstale":"2024-03-16T10:07:13.924724+0000","last_undegraded":"2024-03-16T10:07:13.924724+0000","last_fullsized":"2024-03-16T10:07:13.924724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:41.026293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.191891+0000","last_change":"2024-03-16T10:07:14.199329+0000","last_active":"2024-03-16T10:07:42.191891+0000","last_peered":"2024-03-16T10:07:42.191891+0000","last_clean":"2024-03-16T10:07:42.191891+0000","last_became_active":"2024-03-16T10:07:14.198920+0000","last_became_peered":"2024-03-16T10:07:14.198920+0000","last_unstale":"2024-03-16T10:07:42.191891+0000","last_undegraded":"2024-03-16T10:07:42.191891+0000","last_fullsized":"2024-03-16T10:07:42.191891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:06:32.157083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924734+0000","last_change":"2024-03-16T10:06:08.529441+0000","last_active":"2024-03-16T10:07:13.924734+0000","last_peered":"2024-03-16T10:07:13.924734+0000","last_clean":"2024-03-16T10:07:13.924734+0000","last_became_active":"2024-03-16T10:06:08.528801+0000","last_became_peered":"2024-03-16T10:06:08.528801+0000","last_unstale":"2024-03-16T10:07:13.924734+0000","last_undegraded":"2024-03-16T10:07:13.924734+0000","last_fullsized":"2024-03-16T10:07:13.924734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:50.592415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.929322+0000","last_change":"2024-03-16T10:06:03.236260+0000","last_active":"2024-03-16T10:07:13.929322+0000","last_peered":"2024-03-16T10:07:13.929322+0000","last_clean":"2024-03-16T10:07:13.929322+0000","last_became_active":"2024-03-16T10:06:03.235792+0000","last_became_peered":"2024-03-16T10:06:03.235792+0000","last_unstale":"2024-03-16T10:07:13.929322+0000","last_undegraded":"2024-03-16T10:07:13.929322+0000","last_fullsized":"2024-03-16T10:07:13.929322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:11:59.364244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210511+0000","last_change":"2024-03-16T10:06:04.226962+0000","last_active":"2024-03-16T10:07:42.210511+0000","last_peered":"2024-03-16T10:07:42.210511+0000","last_clean":"2024-03-16T10:07:42.210511+0000","last_became_active":"2024-03-16T10:06:04.226668+0000","last_became_peered":"2024-03-16T10:06:04.226668+0000","last_unstale":"2024-03-16T10:07:42.210511+0000","last_undegraded":"2024-03-16T10:07:42.210511+0000","last_fullsized":"2024-03-16T10:07:42.210511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:53.097834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198018+0000","last_change":"2024-03-16T10:06:08.343528+0000","last_active":"2024-03-16T10:07:14.198018+0000","last_peered":"2024-03-16T10:07:14.198018+0000","last_clean":"2024-03-16T10:07:14.198018+0000","last_became_active":"2024-03-16T10:06:08.340969+0000","last_became_peered":"2024-03-16T10:06:08.340969+0000","last_unstale":"2024-03-16T10:07:14.198018+0000","last_undegraded":"2024-03-16T10:07:14.198018+0000","last_fullsized":"2024-03-16T10:07:14.198018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:54:33.028756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.717346+0000","last_change":"2024-03-16T10:06:03.236166+0000","last_active":"2024-03-16T10:08:00.717346+0000","last_peered":"2024-03-16T10:08:00.717346+0000","last_clean":"2024-03-16T10:08:00.717346+0000","last_became_active":"2024-03-16T10:06:03.235825+0000","last_became_peered":"2024-03-16T10:06:03.235825+0000","last_unstale":"2024-03-16T10:08:00.717346+0000","last_undegraded":"2024-03-16T10:08:00.717346+0000","last_fullsized":"2024-03-16T10:08:00.717346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:39:34.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.208431+0000","last_change":"2024-03-16T10:06:04.220002+0000","last_active":"2024-03-16T10:07:42.208431+0000","last_peered":"2024-03-16T10:07:42.208431+0000","last_clean":"2024-03-16T10:07:42.208431+0000","last_became_active":"2024-03-16T10:06:04.219623+0000","last_became_peered":"2024-03-16T10:06:04.219623+0000","last_unstale":"2024-03-16T10:07:42.208431+0000","last_undegraded":"2024-03-16T10:07:42.208431+0000","last_fullsized":"2024-03-16T10:07:42.208431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:39:35.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.197936+0000","last_change":"2024-03-16T10:06:08.530396+0000","last_active":"2024-03-16T10:07:14.197936+0000","last_peered":"2024-03-16T10:07:14.197936+0000","last_clean":"2024-03-16T10:07:14.197936+0000","last_became_active":"2024-03-16T10:06:08.529102+0000","last_became_peered":"2024-03-16T10:06:08.529102+0000","last_unstale":"2024-03-16T10:07:14.197936+0000","last_undegraded":"2024-03-16T10:07:14.197936+0000","last_fullsized":"2024-03-16T10:07:14.197936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:36.408884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.194154+0000","last_change":"2024-03-16T10:07:13.964902+0000","last_active":"2024-03-16T10:07:42.194154+0000","last_peered":"2024-03-16T10:07:42.194154+0000","last_clean":"2024-03-16T10:07:42.194154+0000","last_became_active":"2024-03-16T10:07:13.963970+0000","last_became_peered":"2024-03-16T10:07:13.963970+0000","last_unstale":"2024-03-16T10:07:42.194154+0000","last_undegraded":"2024-03-16T10:07:42.194154+0000","last_fullsized":"2024-03-16T10:07:42.194154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:15.626355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.753374+0000","last_change":"2024-03-16T10:06:03.247109+0000","last_active":"2024-03-16T10:08:00.753374+0000","last_peered":"2024-03-16T10:08:00.753374+0000","last_clean":"2024-03-16T10:08:00.753374+0000","last_became_active":"2024-03-16T10:06:03.246515+0000","last_became_peered":"2024-03-16T10:06:03.246515+0000","last_unstale":"2024-03-16T10:08:00.753374+0000","last_undegraded":"2024-03-16T10:08:00.753374+0000","last_fullsized":"2024-03-16T10:08:00.753374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:29.744828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926737+0000","last_change":"2024-03-16T10:06:08.334587+0000","last_active":"2024-03-16T10:07:13.926737+0000","last_peered":"2024-03-16T10:07:13.926737+0000","last_clean":"2024-03-16T10:07:13.926737+0000","last_became_active":"2024-03-16T10:06:08.334248+0000","last_became_peered":"2024-03-16T10:06:08.334248+0000","last_unstale":"2024-03-16T10:07:13.926737+0000","last_undegraded":"2024-03-16T10:07:13.926737+0000","last_fullsized":"2024-03-16T10:07:13.926737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:33:28.909540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212641+0000","last_change":"2024-03-16T10:06:04.221069+0000","last_active":"2024-03-16T10:07:42.212641+0000","last_peered":"2024-03-16T10:07:42.212641+0000","last_clean":"2024-03-16T10:07:42.212641+0000","last_became_active":"2024-03-16T10:06:04.218197+0000","last_became_peered":"2024-03-16T10:06:04.218197+0000","last_unstale":"2024-03-16T10:07:42.212641+0000","last_undegraded":"2024-03-16T10:07:42.212641+0000","last_fullsized":"2024-03-16T10:07:42.212641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:38:25.262182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963402+0000","last_change":"2024-03-16T10:06:03.221284+0000","last_active":"2024-03-16T10:07:13.963402+0000","last_peered":"2024-03-16T10:07:13.963402+0000","last_clean":"2024-03-16T10:07:13.963402+0000","last_became_active":"2024-03-16T10:06:03.220467+0000","last_became_peered":"2024-03-16T10:06:03.220467+0000","last_unstale":"2024-03-16T10:07:13.963402+0000","last_undegraded":"2024-03-16T10:07:13.963402+0000","last_fullsized":"2024-03-16T10:07:13.963402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:31:19.753996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926333+0000","last_change":"2024-03-16T10:04:45.615656+0000","last_active":"2024-03-16T10:07:13.926333+0000","last_peered":"2024-03-16T10:07:13.926333+0000","last_clean":"2024-03-16T10:07:13.926333+0000","last_became_active":"2024-03-16T10:04:45.615261+0000","last_became_peered":"2024-03-16T10:04:45.615261+0000","last_unstale":"2024-03-16T10:07:13.926333+0000","last_undegraded":"2024-03-16T10:07:13.926333+0000","last_fullsized":"2024-03-16T10:07:13.926333+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_clean_scrub_stamp":"2024-03-16T10:00:56.880115+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:06.546840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196230+0000","last_change":"2024-03-16T10:06:08.529863+0000","last_active":"2024-03-16T10:07:14.196230+0000","last_peered":"2024-03-16T10:07:14.196230+0000","last_clean":"2024-03-16T10:07:14.196230+0000","last_became_active":"2024-03-16T10:06:08.528934+0000","last_became_peered":"2024-03-16T10:06:08.528934+0000","last_unstale":"2024-03-16T10:07:14.196230+0000","last_undegraded":"2024-03-16T10:07:14.196230+0000","last_fullsized":"2024-03-16T10:07:14.196230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:40.443983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193428+0000","last_change":"2024-03-16T10:06:04.206776+0000","last_active":"2024-03-16T10:07:42.193428+0000","last_peered":"2024-03-16T10:07:42.193428+0000","last_clean":"2024-03-16T10:07:42.193428+0000","last_became_active":"2024-03-16T10:06:04.206333+0000","last_became_peered":"2024-03-16T10:06:04.206333+0000","last_unstale":"2024-03-16T10:07:42.193428+0000","last_undegraded":"2024-03-16T10:07:42.193428+0000","last_fullsized":"2024-03-16T10:07:42.193428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:00:34.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926410+0000","last_change":"2024-03-16T10:06:03.226921+0000","last_active":"2024-03-16T10:07:13.926410+0000","last_peered":"2024-03-16T10:07:13.926410+0000","last_clean":"2024-03-16T10:07:13.926410+0000","last_became_active":"2024-03-16T10:06:03.226596+0000","last_became_peered":"2024-03-16T10:06:03.226596+0000","last_unstale":"2024-03-16T10:07:13.926410+0000","last_undegraded":"2024-03-16T10:07:13.926410+0000","last_fullsized":"2024-03-16T10:07:13.926410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:03:50.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.925983+0000","last_change":"2024-03-16T10:06:08.344432+0000","last_active":"2024-03-16T10:07:13.925983+0000","last_peered":"2024-03-16T10:07:13.925983+0000","last_clean":"2024-03-16T10:07:13.925983+0000","last_became_active":"2024-03-16T10:06:08.343983+0000","last_became_peered":"2024-03-16T10:06:08.343983+0000","last_unstale":"2024-03-16T10:07:13.925983+0000","last_undegraded":"2024-03-16T10:07:13.925983+0000","last_fullsized":"2024-03-16T10:07:13.925983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:41.686700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924317+0000","last_change":"2024-03-16T10:06:03.232806+0000","last_active":"2024-03-16T10:07:13.924317+0000","last_peered":"2024-03-16T10:07:13.924317+0000","last_clean":"2024-03-16T10:07:13.924317+0000","last_became_active":"2024-03-16T10:06:03.232465+0000","last_became_peered":"2024-03-16T10:06:03.232465+0000","last_unstale":"2024-03-16T10:07:13.924317+0000","last_undegraded":"2024-03-16T10:07:13.924317+0000","last_fullsized":"2024-03-16T10:07:13.924317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:36.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.200103+0000","last_change":"2024-03-16T10:06:04.215899+0000","last_active":"2024-03-16T10:07:42.200103+0000","last_peered":"2024-03-16T10:07:42.200103+0000","last_clean":"2024-03-16T10:07:42.200103+0000","last_became_active":"2024-03-16T10:06:04.215474+0000","last_became_peered":"2024-03-16T10:06:04.215474+0000","last_unstale":"2024-03-16T10:07:42.200103+0000","last_undegraded":"2024-03-16T10:07:42.200103+0000","last_fullsized":"2024-03-16T10:07:42.200103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:27:26.663702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.062701+0000","last_change":"2024-03-16T10:06:08.530852+0000","last_active":"2024-03-16T10:07:14.062701+0000","last_peered":"2024-03-16T10:07:14.062701+0000","last_clean":"2024-03-16T10:07:14.062701+0000","last_became_active":"2024-03-16T10:06:08.529386+0000","last_became_peered":"2024-03-16T10:06:08.529386+0000","last_unstale":"2024-03-16T10:07:14.062701+0000","last_undegraded":"2024-03-16T10:07:14.062701+0000","last_fullsized":"2024-03-16T10:07:14.062701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:37:17.201323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.929540+0000","last_change":"2024-03-16T10:06:03.244012+0000","last_active":"2024-03-16T10:07:13.929540+0000","last_peered":"2024-03-16T10:07:13.929540+0000","last_clean":"2024-03-16T10:07:13.929540+0000","last_became_active":"2024-03-16T10:06:03.243612+0000","last_became_peered":"2024-03-16T10:06:03.243612+0000","last_unstale":"2024-03-16T10:07:13.929540+0000","last_undegraded":"2024-03-16T10:07:13.929540+0000","last_fullsized":"2024-03-16T10:07:13.929540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:32:34.807111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.208446+0000","last_change":"2024-03-16T10:06:04.226678+0000","last_active":"2024-03-16T10:07:42.208446+0000","last_peered":"2024-03-16T10:07:42.208446+0000","last_clean":"2024-03-16T10:07:42.208446+0000","last_became_active":"2024-03-16T10:06:04.226214+0000","last_became_peered":"2024-03-16T10:06:04.226214+0000","last_unstale":"2024-03-16T10:07:42.208446+0000","last_undegraded":"2024-03-16T10:07:42.208446+0000","last_fullsized":"2024-03-16T10:07:42.208446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:05.732310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927156+0000","last_change":"2024-03-16T10:06:08.326985+0000","last_active":"2024-03-16T10:07:13.927156+0000","last_peered":"2024-03-16T10:07:13.927156+0000","last_clean":"2024-03-16T10:07:13.927156+0000","last_became_active":"2024-03-16T10:06:08.326429+0000","last_became_peered":"2024-03-16T10:06:08.326429+0000","last_unstale":"2024-03-16T10:07:13.927156+0000","last_undegraded":"2024-03-16T10:07:13.927156+0000","last_fullsized":"2024-03-16T10:07:13.927156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:44:59.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963582+0000","last_change":"2024-03-16T10:06:03.232910+0000","last_active":"2024-03-16T10:07:13.963582+0000","last_peered":"2024-03-16T10:07:13.963582+0000","last_clean":"2024-03-16T10:07:13.963582+0000","last_became_active":"2024-03-16T10:06:03.232682+0000","last_became_peered":"2024-03-16T10:06:03.232682+0000","last_unstale":"2024-03-16T10:07:13.963582+0000","last_undegraded":"2024-03-16T10:07:13.963582+0000","last_fullsized":"2024-03-16T10:07:13.963582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:32:56.731620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.215888+0000","last_change":"2024-03-16T10:06:04.227189+0000","last_active":"2024-03-16T10:07:42.215888+0000","last_peered":"2024-03-16T10:07:42.215888+0000","last_clean":"2024-03-16T10:07:42.215888+0000","last_became_active":"2024-03-16T10:06:04.226723+0000","last_became_peered":"2024-03-16T10:06:04.226723+0000","last_unstale":"2024-03-16T10:07:42.215888+0000","last_undegraded":"2024-03-16T10:07:42.215888+0000","last_fullsized":"2024-03-16T10:07:42.215888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:47.873938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963191+0000","last_change":"2024-03-16T10:06:08.529316+0000","last_active":"2024-03-16T10:07:13.963191+0000","last_peered":"2024-03-16T10:07:13.963191+0000","last_clean":"2024-03-16T10:07:13.963191+0000","last_became_active":"2024-03-16T10:06:08.528903+0000","last_became_peered":"2024-03-16T10:06:08.528903+0000","last_unstale":"2024-03-16T10:07:13.963191+0000","last_undegraded":"2024-03-16T10:07:13.963191+0000","last_fullsized":"2024-03-16T10:07:13.963191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:19:33.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928101+0000","last_change":"2024-03-16T10:06:03.233819+0000","last_active":"2024-03-16T10:07:13.928101+0000","last_peered":"2024-03-16T10:07:13.928101+0000","last_clean":"2024-03-16T10:07:13.928101+0000","last_became_active":"2024-03-16T10:06:03.233593+0000","last_became_peered":"2024-03-16T10:06:03.233593+0000","last_unstale":"2024-03-16T10:07:13.928101+0000","last_undegraded":"2024-03-16T10:07:13.928101+0000","last_fullsized":"2024-03-16T10:07:13.928101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:18.847248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.165905+0000","last_change":"2024-03-16T10:06:04.192682+0000","last_active":"2024-03-16T10:07:42.165905+0000","last_peered":"2024-03-16T10:07:42.165905+0000","last_clean":"2024-03-16T10:07:42.165905+0000","last_became_active":"2024-03-16T10:06:04.192305+0000","last_became_peered":"2024-03-16T10:06:04.192305+0000","last_unstale":"2024-03-16T10:07:42.165905+0000","last_undegraded":"2024-03-16T10:07:42.165905+0000","last_fullsized":"2024-03-16T10:07:42.165905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:46.718663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.930770+0000","last_change":"2024-03-16T10:06:08.355342+0000","last_active":"2024-03-16T10:07:13.930770+0000","last_peered":"2024-03-16T10:07:13.930770+0000","last_clean":"2024-03-16T10:07:13.930770+0000","last_became_active":"2024-03-16T10:06:08.351547+0000","last_became_peered":"2024-03-16T10:06:08.351547+0000","last_unstale":"2024-03-16T10:07:13.930770+0000","last_undegraded":"2024-03-16T10:07:13.930770+0000","last_fullsized":"2024-03-16T10:07:13.930770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:01:00.777949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.062906+0000","last_change":"2024-03-16T10:06:03.242248+0000","last_active":"2024-03-16T10:07:14.062906+0000","last_peered":"2024-03-16T10:07:14.062906+0000","last_clean":"2024-03-16T10:07:14.062906+0000","last_became_active":"2024-03-16T10:06:03.241842+0000","last_became_peered":"2024-03-16T10:06:03.241842+0000","last_unstale":"2024-03-16T10:07:14.062906+0000","last_undegraded":"2024-03-16T10:07:14.062906+0000","last_fullsized":"2024-03-16T10:07:14.062906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:03:35.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193391+0000","last_change":"2024-03-16T10:06:04.221790+0000","last_active":"2024-03-16T10:07:42.193391+0000","last_peered":"2024-03-16T10:07:42.193391+0000","last_clean":"2024-03-16T10:07:42.193391+0000","last_became_active":"2024-03-16T10:06:04.221475+0000","last_became_peered":"2024-03-16T10:06:04.221475+0000","last_unstale":"2024-03-16T10:07:42.193391+0000","last_undegraded":"2024-03-16T10:07:42.193391+0000","last_fullsized":"2024-03-16T10:07:42.193391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:05:50.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963254+0000","last_change":"2024-03-16T10:06:08.337906+0000","last_active":"2024-03-16T10:07:13.963254+0000","last_peered":"2024-03-16T10:07:13.963254+0000","last_clean":"2024-03-16T10:07:13.963254+0000","last_became_active":"2024-03-16T10:06:08.337444+0000","last_became_peered":"2024-03-16T10:06:08.337444+0000","last_unstale":"2024-03-16T10:07:13.963254+0000","last_undegraded":"2024-03-16T10:07:13.963254+0000","last_fullsized":"2024-03-16T10:07:13.963254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:09:03.997606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926632+0000","last_change":"2024-03-16T10:06:03.235717+0000","last_active":"2024-03-16T10:07:13.926632+0000","last_peered":"2024-03-16T10:07:13.926632+0000","last_clean":"2024-03-16T10:07:13.926632+0000","last_became_active":"2024-03-16T10:06:03.235331+0000","last_became_peered":"2024-03-16T10:06:03.235331+0000","last_unstale":"2024-03-16T10:07:13.926632+0000","last_undegraded":"2024-03-16T10:07:13.926632+0000","last_fullsized":"2024-03-16T10:07:13.926632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:08.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214287+0000","last_change":"2024-03-16T10:06:04.229219+0000","last_active":"2024-03-16T10:07:42.214287+0000","last_peered":"2024-03-16T10:07:42.214287+0000","last_clean":"2024-03-16T10:07:42.214287+0000","last_became_active":"2024-03-16T10:06:04.226850+0000","last_became_peered":"2024-03-16T10:06:04.226850+0000","last_unstale":"2024-03-16T10:07:42.214287+0000","last_undegraded":"2024-03-16T10:07:42.214287+0000","last_fullsized":"2024-03-16T10:07:42.214287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:22:48.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963311+0000","last_change":"2024-03-16T10:06:08.529201+0000","last_active":"2024-03-16T10:07:13.963311+0000","last_peered":"2024-03-16T10:07:13.963311+0000","last_clean":"2024-03-16T10:07:13.963311+0000","last_became_active":"2024-03-16T10:06:08.528782+0000","last_became_peered":"2024-03-16T10:06:08.528782+0000","last_unstale":"2024-03-16T10:07:13.963311+0000","last_undegraded":"2024-03-16T10:07:13.963311+0000","last_fullsized":"2024-03-16T10:07:13.963311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:35:38.267168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.968463+0000","last_change":"2024-03-16T10:07:13.968463+0000","last_active":"2024-03-16T10:07:13.968463+0000","last_peered":"2024-03-16T10:07:13.968463+0000","last_clean":"2024-03-16T10:07:13.968463+0000","last_became_active":"2024-03-16T10:07:13.964144+0000","last_became_peered":"2024-03-16T10:07:13.964144+0000","last_unstale":"2024-03-16T10:07:13.968463+0000","last_undegraded":"2024-03-16T10:07:13.968463+0000","last_fullsized":"2024-03-16T10:07:13.968463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:30:16.852944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210780+0000","last_change":"2024-03-16T10:06:04.217699+0000","last_active":"2024-03-16T10:07:42.210780+0000","last_peered":"2024-03-16T10:07:42.210780+0000","last_clean":"2024-03-16T10:07:42.210780+0000","last_became_active":"2024-03-16T10:06:04.217378+0000","last_became_peered":"2024-03-16T10:06:04.217378+0000","last_unstale":"2024-03-16T10:07:42.210780+0000","last_undegraded":"2024-03-16T10:07:42.210780+0000","last_fullsized":"2024-03-16T10:07:42.210780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:35:23.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928126+0000","last_change":"2024-03-16T10:06:08.344488+0000","last_active":"2024-03-16T10:07:13.928126+0000","last_peered":"2024-03-16T10:07:13.928126+0000","last_clean":"2024-03-16T10:07:13.928126+0000","last_became_active":"2024-03-16T10:06:08.343964+0000","last_became_peered":"2024-03-16T10:06:08.343964+0000","last_unstale":"2024-03-16T10:07:13.928126+0000","last_undegraded":"2024-03-16T10:07:13.928126+0000","last_fullsized":"2024-03-16T10:07:13.928126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:41:09.144062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:59.722296+0000","last_change":"2024-03-16T10:06:03.231805+0000","last_active":"2024-03-16T10:07:59.722296+0000","last_peered":"2024-03-16T10:07:59.722296+0000","last_clean":"2024-03-16T10:07:59.722296+0000","last_became_active":"2024-03-16T10:06:03.230605+0000","last_became_peered":"2024-03-16T10:06:03.230605+0000","last_unstale":"2024-03-16T10:07:59.722296+0000","last_undegraded":"2024-03-16T10:07:59.722296+0000","last_fullsized":"2024-03-16T10:07:59.722296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:21.545277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211237+0000","last_change":"2024-03-16T10:06:04.195674+0000","last_active":"2024-03-16T10:07:42.211237+0000","last_peered":"2024-03-16T10:07:42.211237+0000","last_clean":"2024-03-16T10:07:42.211237+0000","last_became_active":"2024-03-16T10:06:04.195364+0000","last_became_peered":"2024-03-16T10:06:04.195364+0000","last_unstale":"2024-03-16T10:07:42.211237+0000","last_undegraded":"2024-03-16T10:07:42.211237+0000","last_fullsized":"2024-03-16T10:07:42.211237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:40.101280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.933064+0000","last_change":"2024-03-16T10:06:08.342352+0000","last_active":"2024-03-16T10:07:13.933064+0000","last_peered":"2024-03-16T10:07:13.933064+0000","last_clean":"2024-03-16T10:07:13.933064+0000","last_became_active":"2024-03-16T10:06:08.341910+0000","last_became_peered":"2024-03-16T10:06:08.341910+0000","last_unstale":"2024-03-16T10:07:13.933064+0000","last_undegraded":"2024-03-16T10:07:13.933064+0000","last_fullsized":"2024-03-16T10:07:13.933064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:23.446076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196016+0000","last_change":"2024-03-16T10:06:03.236814+0000","last_active":"2024-03-16T10:07:14.196016+0000","last_peered":"2024-03-16T10:07:14.196016+0000","last_clean":"2024-03-16T10:07:14.196016+0000","last_became_active":"2024-03-16T10:06:03.236428+0000","last_became_peered":"2024-03-16T10:06:03.236428+0000","last_unstale":"2024-03-16T10:07:14.196016+0000","last_undegraded":"2024-03-16T10:07:14.196016+0000","last_fullsized":"2024-03-16T10:07:14.196016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:44.565924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.197506+0000","last_change":"2024-03-16T10:06:04.224725+0000","last_active":"2024-03-16T10:07:42.197506+0000","last_peered":"2024-03-16T10:07:42.197506+0000","last_clean":"2024-03-16T10:07:42.197506+0000","last_became_active":"2024-03-16T10:06:04.224308+0000","last_became_peered":"2024-03-16T10:06:04.224308+0000","last_unstale":"2024-03-16T10:07:42.197506+0000","last_undegraded":"2024-03-16T10:07:42.197506+0000","last_fullsized":"2024-03-16T10:07:42.197506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:55.128000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195273+0000","last_change":"2024-03-16T10:06:08.330089+0000","last_active":"2024-03-16T10:07:14.195273+0000","last_peered":"2024-03-16T10:07:14.195273+0000","last_clean":"2024-03-16T10:07:14.195273+0000","last_became_active":"2024-03-16T10:06:08.329692+0000","last_became_peered":"2024-03-16T10:06:08.329692+0000","last_unstale":"2024-03-16T10:07:14.195273+0000","last_undegraded":"2024-03-16T10:07:14.195273+0000","last_fullsized":"2024-03-16T10:07:14.195273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:22.141351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198145+0000","last_change":"2024-03-16T10:06:03.247525+0000","last_active":"2024-03-16T10:07:14.198145+0000","last_peered":"2024-03-16T10:07:14.198145+0000","last_clean":"2024-03-16T10:07:14.198145+0000","last_became_active":"2024-03-16T10:06:03.246854+0000","last_became_peered":"2024-03-16T10:06:03.246854+0000","last_unstale":"2024-03-16T10:07:14.198145+0000","last_undegraded":"2024-03-16T10:07:14.198145+0000","last_fullsized":"2024-03-16T10:07:14.198145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:20.648750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196941+0000","last_change":"2024-03-16T10:06:04.194982+0000","last_active":"2024-03-16T10:07:42.196941+0000","last_peered":"2024-03-16T10:07:42.196941+0000","last_clean":"2024-03-16T10:07:42.196941+0000","last_became_active":"2024-03-16T10:06:04.194696+0000","last_became_peered":"2024-03-16T10:06:04.194696+0000","last_unstale":"2024-03-16T10:07:42.196941+0000","last_undegraded":"2024-03-16T10:07:42.196941+0000","last_fullsized":"2024-03-16T10:07:42.196941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:20.085100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.192953+0000","last_change":"2024-03-16T10:06:04.196047+0000","last_active":"2024-03-16T10:07:42.192953+0000","last_peered":"2024-03-16T10:07:42.192953+0000","last_clean":"2024-03-16T10:07:42.192953+0000","last_became_active":"2024-03-16T10:06:04.195742+0000","last_became_peered":"2024-03-16T10:06:04.195742+0000","last_unstale":"2024-03-16T10:07:42.192953+0000","last_undegraded":"2024-03-16T10:07:42.192953+0000","last_fullsized":"2024-03-16T10:07:42.192953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:00:54.239827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190649+0000","last_change":"2024-03-16T10:06:04.193481+0000","last_active":"2024-03-16T10:07:42.190649+0000","last_peered":"2024-03-16T10:07:42.190649+0000","last_clean":"2024-03-16T10:07:42.190649+0000","last_became_active":"2024-03-16T10:06:04.193145+0000","last_became_peered":"2024-03-16T10:06:04.193145+0000","last_unstale":"2024-03-16T10:07:42.190649+0000","last_undegraded":"2024-03-16T10:07:42.190649+0000","last_fullsized":"2024-03-16T10:07:42.190649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:02:10.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.209114+0000","last_change":"2024-03-16T10:06:04.223322+0000","last_active":"2024-03-16T10:07:42.209114+0000","last_peered":"2024-03-16T10:07:42.209114+0000","last_clean":"2024-03-16T10:07:42.209114+0000","last_became_active":"2024-03-16T10:06:04.223116+0000","last_became_peered":"2024-03-16T10:06:04.223116+0000","last_unstale":"2024-03-16T10:07:42.209114+0000","last_undegraded":"2024-03-16T10:07:42.209114+0000","last_fullsized":"2024-03-16T10:07:42.209114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:46.641716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216835+0000","last_change":"2024-03-16T10:07:14.199872+0000","last_active":"2024-03-16T10:07:42.216835+0000","last_peered":"2024-03-16T10:07:42.216835+0000","last_clean":"2024-03-16T10:07:42.216835+0000","last_became_active":"2024-03-16T10:07:14.199210+0000","last_became_peered":"2024-03-16T10:07:14.199210+0000","last_unstale":"2024-03-16T10:07:42.216835+0000","last_undegraded":"2024-03-16T10:07:42.216835+0000","last_fullsized":"2024-03-16T10:07:42.216835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:42:36.253729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214690+0000","last_change":"2024-03-16T10:06:04.225518+0000","last_active":"2024-03-16T10:07:42.214690+0000","last_peered":"2024-03-16T10:07:42.214690+0000","last_clean":"2024-03-16T10:07:42.214690+0000","last_became_active":"2024-03-16T10:06:04.225102+0000","last_became_peered":"2024-03-16T10:06:04.225102+0000","last_unstale":"2024-03-16T10:07:42.214690+0000","last_undegraded":"2024-03-16T10:07:42.214690+0000","last_fullsized":"2024-03-16T10:07:42.214690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:38:54.493238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190560+0000","last_change":"2024-03-16T10:06:04.192978+0000","last_active":"2024-03-16T10:07:42.190560+0000","last_peered":"2024-03-16T10:07:42.190560+0000","last_clean":"2024-03-16T10:07:42.190560+0000","last_became_active":"2024-03-16T10:06:04.192641+0000","last_became_peered":"2024-03-16T10:06:04.192641+0000","last_unstale":"2024-03-16T10:07:42.190560+0000","last_undegraded":"2024-03-16T10:07:42.190560+0000","last_fullsized":"2024-03-16T10:07:42.190560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:58.334266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.202684+0000","last_change":"2024-03-16T10:06:04.190020+0000","last_active":"2024-03-16T10:07:42.202684+0000","last_peered":"2024-03-16T10:07:42.202684+0000","last_clean":"2024-03-16T10:07:42.202684+0000","last_became_active":"2024-03-16T10:06:04.189703+0000","last_became_peered":"2024-03-16T10:06:04.189703+0000","last_unstale":"2024-03-16T10:07:42.202684+0000","last_undegraded":"2024-03-16T10:07:42.202684+0000","last_fullsized":"2024-03-16T10:07:42.202684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:23.689558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196634+0000","last_change":"2024-03-16T10:06:04.234714+0000","last_active":"2024-03-16T10:07:42.196634+0000","last_peered":"2024-03-16T10:07:42.196634+0000","last_clean":"2024-03-16T10:07:42.196634+0000","last_became_active":"2024-03-16T10:06:04.233246+0000","last_became_peered":"2024-03-16T10:06:04.233246+0000","last_unstale":"2024-03-16T10:07:42.196634+0000","last_undegraded":"2024-03-16T10:07:42.196634+0000","last_fullsized":"2024-03-16T10:07:42.196634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:13:31.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211762+0000","last_change":"2024-03-16T10:06:04.219242+0000","last_active":"2024-03-16T10:07:42.211762+0000","last_peered":"2024-03-16T10:07:42.211762+0000","last_clean":"2024-03-16T10:07:42.211762+0000","last_became_active":"2024-03-16T10:06:04.218955+0000","last_became_peered":"2024-03-16T10:06:04.218955+0000","last_unstale":"2024-03-16T10:07:42.211762+0000","last_undegraded":"2024-03-16T10:07:42.211762+0000","last_fullsized":"2024-03-16T10:07:42.211762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:40.760143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.203178+0000","last_change":"2024-03-16T10:06:04.231516+0000","last_active":"2024-03-16T10:07:42.203178+0000","last_peered":"2024-03-16T10:07:42.203178+0000","last_clean":"2024-03-16T10:07:42.203178+0000","last_became_active":"2024-03-16T10:06:04.230328+0000","last_became_peered":"2024-03-16T10:06:04.230328+0000","last_unstale":"2024-03-16T10:07:42.203178+0000","last_undegraded":"2024-03-16T10:07:42.203178+0000","last_fullsized":"2024-03-16T10:07:42.203178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:48:55.678921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211721+0000","last_change":"2024-03-16T10:07:13.930195+0000","last_active":"2024-03-16T10:07:42.211721+0000","last_peered":"2024-03-16T10:07:42.211721+0000","last_clean":"2024-03-16T10:07:42.211721+0000","last_became_active":"2024-03-16T10:07:13.929801+0000","last_became_peered":"2024-03-16T10:07:13.929801+0000","last_unstale":"2024-03-16T10:07:42.211721+0000","last_undegraded":"2024-03-16T10:07:42.211721+0000","last_fullsized":"2024-03-16T10:07:42.211721+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:00:07.902956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.215327+0000","last_change":"2024-03-16T10:06:04.226025+0000","last_active":"2024-03-16T10:07:42.215327+0000","last_peered":"2024-03-16T10:07:42.215327+0000","last_clean":"2024-03-16T10:07:42.215327+0000","last_became_active":"2024-03-16T10:06:04.225657+0000","last_became_peered":"2024-03-16T10:06:04.225657+0000","last_unstale":"2024-03-16T10:07:42.215327+0000","last_undegraded":"2024-03-16T10:07:42.215327+0000","last_fullsized":"2024-03-16T10:07:42.215327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:08:15.163902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193956+0000","last_change":"2024-03-16T10:06:04.198047+0000","last_active":"2024-03-16T10:07:42.193956+0000","last_peered":"2024-03-16T10:07:42.193956+0000","last_clean":"2024-03-16T10:07:42.193956+0000","last_became_active":"2024-03-16T10:06:04.197751+0000","last_became_peered":"2024-03-16T10:06:04.197751+0000","last_unstale":"2024-03-16T10:07:42.193956+0000","last_undegraded":"2024-03-16T10:07:42.193956+0000","last_fullsized":"2024-03-16T10:07:42.193956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:52.703845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":215634,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70360,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":847,"num_read_kb":504,"num_write":250,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":211080,"data_compressed":5523,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":505049,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1154627,"data_compressed":58972,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":508431,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":504601,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1101351,"data_compressed":57131,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":499839,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":563743,"data_compressed":22487,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":501827,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":564495,"data_compressed":22487,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":499547,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":504445,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1096401,"data_compressed":57131,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64626,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65378,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65142,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:08:04.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T10:08:04.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:03 smithi039 ceph-mon[31510]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:04.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:03 smithi105 ceph-mon[36559]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:03 smithi194 ceph-mon[36445]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:04.637 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:05.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:04 smithi039 ceph-mon[31510]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:05.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:04 smithi105 ceph-mon[36559]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:05.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:04 smithi194 ceph-mon[36445]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:06.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:05 smithi039 ceph-mon[31510]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-16T10:08:06.265 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:05 smithi105 ceph-mon[36559]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-16T10:08:06.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:05 smithi194 ceph-mon[36445]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-16T10:08:06.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 -- 172.21.15.39:0/1059295083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 msgr2=0x7fb0b810f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:06.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 --2- 172.21.15.39:0/1059295083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b810f160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b0d0 tx=0x7fb0b000b3e0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 -- 172.21.15.39:0/1059295083 shutdown_connections 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 --2- 172.21.15.39:0/1059295083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b810f160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 --2- 172.21.15.39:0/1059295083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 --2- 172.21.15.39:0/1059295083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.841+0000 7fb0be800700 1 -- 172.21.15.39:0/1059295083 >> 172.21.15.39:0/1059295083 conn(0x7fb0b8078100 msgr2=0x7fb0b8078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.842+0000 7fb0be800700 1 -- 172.21.15.39:0/1059295083 shutdown_connections 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.842+0000 7fb0be800700 1 -- 172.21.15.39:0/1059295083 wait complete. 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.843+0000 7fb0be800700 1 Processor -- start 2024-03-16T10:08:06.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.843+0000 7fb0be800700 1 -- start start 2024-03-16T10:08:06.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b81a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b81a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81125c0 con 0x7fb0b8108320 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8112440 con 0x7fb0b8069f80 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0be800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8112740 con 0x7fb0b806a920 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0b7400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b81a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0b6a00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b81a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0b7400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b81a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:38670/0 (socket says 172.21.15.39:38670) 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0b7400700 1 -- 172.21.15.39:0/2101567927 learned_addr learned my addr 172.21.15.39:0/2101567927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:06.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.844+0000 7fb0b7e00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 -- 172.21.15.39:0/2101567927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 msgr2=0x7fb0b81a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b81a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 -- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 msgr2=0x7fb0b819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 -- 172.21.15.39:0/2101567927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b819b210 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b7e00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0b6a00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b81a0e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e9f0 tx=0x7fb0a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a400cbd0 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a400cd70 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b819b500 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.845+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4018710 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.846+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a77c0 con 0x7fb0b806a920 2024-03-16T10:08:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.847+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8068f90 con 0x7fb0b806a920 2024-03-16T10:08:06.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.848+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb0a4010430 con 0x7fb0b806a920 2024-03-16T10:08:06.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.850+0000 7fb0afe00700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 0x7fb09c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:06.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.850+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fb0a40ceff0 con 0x7fb0b806a920 2024-03-16T10:08:06.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.850+0000 7fb0b7400700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 0x7fb09c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:06.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.851+0000 7fb0b7400700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 0x7fb09c0a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb0b819bbf0 tx=0x7fb0a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:06.854+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb0a409bd30 con 0x7fb0b806a920 2024-03-16T10:08:07.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.076+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0b810dbc0 con 0x7fb09c0a2890 2024-03-16T10:08:07.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.100+0000 7fb0afe00700 1 -- 172.21.15.39:0/2101567927 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340784 (secure 0 0 0) 0x7fb0b810dbc0 con 0x7fb09c0a2890 2024-03-16T10:08:07.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:07.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 msgr2=0x7fb09c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 0x7fb09c0a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb0b819bbf0 tx=0x7fb0a8009260 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 msgr2=0x7fb0b81a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b81a0e30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e9f0 tx=0x7fb0a400ed00 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 shutdown_connections 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fb09c0a2890 0x7fb09c0a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0b8108320 0x7fb0b819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb0b8069f80 0x7fb0b81a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 --2- 172.21.15.39:0/2101567927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0b806a920 0x7fb0b81a0e30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.107+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 >> 172.21.15.39:0/2101567927 conn(0x7fb0b8078100 msgr2=0x7fb0b810d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.108+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 shutdown_connections 2024-03-16T10:08:07.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:07.108+0000 7fb0be800700 1 -- 172.21.15.39:0/2101567927 wait complete. 2024-03-16T10:08:07.111 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-03-16T10:08:07.987 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-03-16T10:08:06.252057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660962,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":998,"num_read_kb":623,"num_write":433,"num_write_kb":3087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":857,"ondisk_log_size":857,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353224,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588600,"statfs":{"total":1151940427776,"available":1151578726400,"internally_reserved":0,"allocated":12689408,"data_stored":8004356,"data_compressed":362730,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19140,"internal_metadata":348894524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005573"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216684+0000","last_change":"2024-03-16T10:06:04.221459+0000","last_active":"2024-03-16T10:07:42.216684+0000","last_peered":"2024-03-16T10:07:42.216684+0000","last_clean":"2024-03-16T10:07:42.216684+0000","last_became_active":"2024-03-16T10:06:04.221172+0000","last_became_peered":"2024-03-16T10:06:04.221172+0000","last_unstale":"2024-03-16T10:07:42.216684+0000","last_undegraded":"2024-03-16T10:07:42.216684+0000","last_fullsized":"2024-03-16T10:07:42.216684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:58:28.019985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213582+0000","last_change":"2024-03-16T10:06:04.205876+0000","last_active":"2024-03-16T10:07:42.213582+0000","last_peered":"2024-03-16T10:07:42.213582+0000","last_clean":"2024-03-16T10:07:42.213582+0000","last_became_active":"2024-03-16T10:06:04.205473+0000","last_became_peered":"2024-03-16T10:06:04.205473+0000","last_unstale":"2024-03-16T10:07:42.213582+0000","last_undegraded":"2024-03-16T10:07:42.213582+0000","last_fullsized":"2024-03-16T10:07:42.213582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:37.981148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217024+0000","last_change":"2024-03-16T10:06:04.197872+0000","last_active":"2024-03-16T10:07:42.217024+0000","last_peered":"2024-03-16T10:07:42.217024+0000","last_clean":"2024-03-16T10:07:42.217024+0000","last_became_active":"2024-03-16T10:06:04.197347+0000","last_became_peered":"2024-03-16T10:06:04.197347+0000","last_unstale":"2024-03-16T10:07:42.217024+0000","last_undegraded":"2024-03-16T10:07:42.217024+0000","last_fullsized":"2024-03-16T10:07:42.217024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:37.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.191502+0000","last_change":"2024-03-16T10:06:04.196359+0000","last_active":"2024-03-16T10:07:42.191502+0000","last_peered":"2024-03-16T10:07:42.191502+0000","last_clean":"2024-03-16T10:07:42.191502+0000","last_became_active":"2024-03-16T10:06:04.196072+0000","last_became_peered":"2024-03-16T10:06:04.196072+0000","last_unstale":"2024-03-16T10:07:42.191502+0000","last_undegraded":"2024-03-16T10:07:42.191502+0000","last_fullsized":"2024-03-16T10:07:42.191502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:03.641536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196468+0000","last_change":"2024-03-16T10:07:14.198887+0000","last_active":"2024-03-16T10:07:42.196468+0000","last_peered":"2024-03-16T10:07:42.196468+0000","last_clean":"2024-03-16T10:07:42.196468+0000","last_became_active":"2024-03-16T10:07:14.198554+0000","last_became_peered":"2024-03-16T10:07:14.198554+0000","last_unstale":"2024-03-16T10:07:42.196468+0000","last_undegraded":"2024-03-16T10:07:42.196468+0000","last_fullsized":"2024-03-16T10:07:42.196468+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:25.665817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196080+0000","last_change":"2024-03-16T10:06:04.214055+0000","last_active":"2024-03-16T10:07:42.196080+0000","last_peered":"2024-03-16T10:07:42.196080+0000","last_clean":"2024-03-16T10:07:42.196080+0000","last_became_active":"2024-03-16T10:06:04.213792+0000","last_became_peered":"2024-03-16T10:06:04.213792+0000","last_unstale":"2024-03-16T10:07:42.196080+0000","last_undegraded":"2024-03-16T10:07:42.196080+0000","last_fullsized":"2024-03-16T10:07:42.196080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:50.142741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198356+0000","last_change":"2024-03-16T10:06:04.208874+0000","last_active":"2024-03-16T10:07:42.198356+0000","last_peered":"2024-03-16T10:07:42.198356+0000","last_clean":"2024-03-16T10:07:42.198356+0000","last_became_active":"2024-03-16T10:06:04.208553+0000","last_became_peered":"2024-03-16T10:06:04.208553+0000","last_unstale":"2024-03-16T10:07:42.198356+0000","last_undegraded":"2024-03-16T10:07:42.198356+0000","last_fullsized":"2024-03-16T10:07:42.198356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:33.885089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214693+0000","last_change":"2024-03-16T10:06:04.233487+0000","last_active":"2024-03-16T10:07:42.214693+0000","last_peered":"2024-03-16T10:07:42.214693+0000","last_clean":"2024-03-16T10:07:42.214693+0000","last_became_active":"2024-03-16T10:06:04.228530+0000","last_became_peered":"2024-03-16T10:06:04.228530+0000","last_unstale":"2024-03-16T10:07:42.214693+0000","last_undegraded":"2024-03-16T10:07:42.214693+0000","last_fullsized":"2024-03-16T10:07:42.214693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:53:12.270229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.209815+0000","last_change":"2024-03-16T10:06:04.190965+0000","last_active":"2024-03-16T10:07:42.209815+0000","last_peered":"2024-03-16T10:07:42.209815+0000","last_clean":"2024-03-16T10:07:42.209815+0000","last_became_active":"2024-03-16T10:06:04.190681+0000","last_became_peered":"2024-03-16T10:06:04.190681+0000","last_unstale":"2024-03-16T10:07:42.209815+0000","last_undegraded":"2024-03-16T10:07:42.209815+0000","last_fullsized":"2024-03-16T10:07:42.209815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:24:46.498680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196408+0000","last_change":"2024-03-16T10:06:04.198098+0000","last_active":"2024-03-16T10:07:42.196408+0000","last_peered":"2024-03-16T10:07:42.196408+0000","last_clean":"2024-03-16T10:07:42.196408+0000","last_became_active":"2024-03-16T10:06:04.197758+0000","last_became_peered":"2024-03-16T10:06:04.197758+0000","last_unstale":"2024-03-16T10:07:42.196408+0000","last_undegraded":"2024-03-16T10:07:42.196408+0000","last_fullsized":"2024-03-16T10:07:42.196408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:27.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211115+0000","last_change":"2024-03-16T10:06:04.195970+0000","last_active":"2024-03-16T10:07:42.211115+0000","last_peered":"2024-03-16T10:07:42.211115+0000","last_clean":"2024-03-16T10:07:42.211115+0000","last_became_active":"2024-03-16T10:06:04.195522+0000","last_became_peered":"2024-03-16T10:06:04.195522+0000","last_unstale":"2024-03-16T10:07:42.211115+0000","last_undegraded":"2024-03-16T10:07:42.211115+0000","last_fullsized":"2024-03-16T10:07:42.211115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:22.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190896+0000","last_change":"2024-03-16T10:06:04.196075+0000","last_active":"2024-03-16T10:07:42.190896+0000","last_peered":"2024-03-16T10:07:42.190896+0000","last_clean":"2024-03-16T10:07:42.190896+0000","last_became_active":"2024-03-16T10:06:04.195785+0000","last_became_peered":"2024-03-16T10:06:04.195785+0000","last_unstale":"2024-03-16T10:07:42.190896+0000","last_undegraded":"2024-03-16T10:07:42.190896+0000","last_fullsized":"2024-03-16T10:07:42.190896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:52:50.016787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196307+0000","last_change":"2024-03-16T10:06:04.206173+0000","last_active":"2024-03-16T10:07:42.196307+0000","last_peered":"2024-03-16T10:07:42.196307+0000","last_clean":"2024-03-16T10:07:42.196307+0000","last_became_active":"2024-03-16T10:06:04.205766+0000","last_became_peered":"2024-03-16T10:06:04.205766+0000","last_unstale":"2024-03-16T10:07:42.196307+0000","last_undegraded":"2024-03-16T10:07:42.196307+0000","last_fullsized":"2024-03-16T10:07:42.196307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:25:47.405044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213854+0000","last_change":"2024-03-16T10:06:04.196204+0000","last_active":"2024-03-16T10:07:42.213854+0000","last_peered":"2024-03-16T10:07:42.213854+0000","last_clean":"2024-03-16T10:07:42.213854+0000","last_became_active":"2024-03-16T10:06:04.194255+0000","last_became_peered":"2024-03-16T10:06:04.194255+0000","last_unstale":"2024-03-16T10:07:42.213854+0000","last_undegraded":"2024-03-16T10:07:42.213854+0000","last_fullsized":"2024-03-16T10:07:42.213854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:44.367876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214098+0000","last_change":"2024-03-16T10:06:04.203298+0000","last_active":"2024-03-16T10:07:42.214098+0000","last_peered":"2024-03-16T10:07:42.214098+0000","last_clean":"2024-03-16T10:07:42.214098+0000","last_became_active":"2024-03-16T10:06:04.202790+0000","last_became_peered":"2024-03-16T10:06:04.202790+0000","last_unstale":"2024-03-16T10:07:42.214098+0000","last_undegraded":"2024-03-16T10:07:42.214098+0000","last_fullsized":"2024-03-16T10:07:42.214098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:27.360861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217160+0000","last_change":"2024-03-16T10:06:04.191196+0000","last_active":"2024-03-16T10:07:42.217160+0000","last_peered":"2024-03-16T10:07:42.217160+0000","last_clean":"2024-03-16T10:07:42.217160+0000","last_became_active":"2024-03-16T10:06:04.190873+0000","last_became_peered":"2024-03-16T10:06:04.190873+0000","last_unstale":"2024-03-16T10:07:42.217160+0000","last_undegraded":"2024-03-16T10:07:42.217160+0000","last_fullsized":"2024-03-16T10:07:42.217160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:09:59.272388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216824+0000","last_change":"2024-03-16T10:06:04.221443+0000","last_active":"2024-03-16T10:07:42.216824+0000","last_peered":"2024-03-16T10:07:42.216824+0000","last_clean":"2024-03-16T10:07:42.216824+0000","last_became_active":"2024-03-16T10:06:04.221182+0000","last_became_peered":"2024-03-16T10:06:04.221182+0000","last_unstale":"2024-03-16T10:07:42.216824+0000","last_undegraded":"2024-03-16T10:07:42.216824+0000","last_fullsized":"2024-03-16T10:07:42.216824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:04:08.139784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198435+0000","last_change":"2024-03-16T10:06:04.208876+0000","last_active":"2024-03-16T10:07:42.198435+0000","last_peered":"2024-03-16T10:07:42.198435+0000","last_clean":"2024-03-16T10:07:42.198435+0000","last_became_active":"2024-03-16T10:06:04.208361+0000","last_became_peered":"2024-03-16T10:06:04.208361+0000","last_unstale":"2024-03-16T10:07:42.198435+0000","last_undegraded":"2024-03-16T10:07:42.198435+0000","last_fullsized":"2024-03-16T10:07:42.198435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:53.156832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211593+0000","last_change":"2024-03-16T10:06:04.228080+0000","last_active":"2024-03-16T10:07:42.211593+0000","last_peered":"2024-03-16T10:07:42.211593+0000","last_clean":"2024-03-16T10:07:42.211593+0000","last_became_active":"2024-03-16T10:06:04.227799+0000","last_became_peered":"2024-03-16T10:06:04.227799+0000","last_unstale":"2024-03-16T10:07:42.211593+0000","last_undegraded":"2024-03-16T10:07:42.211593+0000","last_fullsized":"2024-03-16T10:07:42.211593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:40:55.113354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927329+0000","last_change":"2024-03-16T10:06:08.349744+0000","last_active":"2024-03-16T10:07:13.927329+0000","last_peered":"2024-03-16T10:07:13.927329+0000","last_clean":"2024-03-16T10:07:13.927329+0000","last_became_active":"2024-03-16T10:06:08.349359+0000","last_became_peered":"2024-03-16T10:06:08.349359+0000","last_unstale":"2024-03-16T10:07:13.927329+0000","last_undegraded":"2024-03-16T10:07:13.927329+0000","last_fullsized":"2024-03-16T10:07:13.927329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:30:02.614189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":122,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:59.715655+0000","last_change":"2024-03-16T10:06:03.234836+0000","last_active":"2024-03-16T10:07:59.715655+0000","last_peered":"2024-03-16T10:07:59.715655+0000","last_clean":"2024-03-16T10:07:59.715655+0000","last_became_active":"2024-03-16T10:06:03.234436+0000","last_became_peered":"2024-03-16T10:06:03.234436+0000","last_unstale":"2024-03-16T10:07:59.715655+0000","last_undegraded":"2024-03-16T10:07:59.715655+0000","last_fullsized":"2024-03-16T10:07:59.715655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:52:34.069875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.206679+0000","last_change":"2024-03-16T10:06:04.222012+0000","last_active":"2024-03-16T10:07:42.206679+0000","last_peered":"2024-03-16T10:07:42.206679+0000","last_clean":"2024-03-16T10:07:42.206679+0000","last_became_active":"2024-03-16T10:06:04.221661+0000","last_became_peered":"2024-03-16T10:06:04.221661+0000","last_unstale":"2024-03-16T10:07:42.206679+0000","last_undegraded":"2024-03-16T10:07:42.206679+0000","last_fullsized":"2024-03-16T10:07:42.206679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:07.837752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.962913+0000","last_change":"2024-03-16T10:06:08.345658+0000","last_active":"2024-03-16T10:07:13.962913+0000","last_peered":"2024-03-16T10:07:13.962913+0000","last_clean":"2024-03-16T10:07:13.962913+0000","last_became_active":"2024-03-16T10:06:08.345291+0000","last_became_peered":"2024-03-16T10:06:08.345291+0000","last_unstale":"2024-03-16T10:07:13.962913+0000","last_undegraded":"2024-03-16T10:07:13.962913+0000","last_fullsized":"2024-03-16T10:07:13.962913+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:43:39.734164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.064474+0000","last_change":"2024-03-16T10:06:03.244368+0000","last_active":"2024-03-16T10:07:14.064474+0000","last_peered":"2024-03-16T10:07:14.064474+0000","last_clean":"2024-03-16T10:07:14.064474+0000","last_became_active":"2024-03-16T10:06:03.243369+0000","last_became_peered":"2024-03-16T10:06:03.243369+0000","last_unstale":"2024-03-16T10:07:14.064474+0000","last_undegraded":"2024-03-16T10:07:14.064474+0000","last_fullsized":"2024-03-16T10:07:14.064474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:38.473988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.217354+0000","last_change":"2024-03-16T10:06:04.217255+0000","last_active":"2024-03-16T10:07:42.217354+0000","last_peered":"2024-03-16T10:07:42.217354+0000","last_clean":"2024-03-16T10:07:42.217354+0000","last_became_active":"2024-03-16T10:06:04.216659+0000","last_became_peered":"2024-03-16T10:06:04.216659+0000","last_unstale":"2024-03-16T10:07:42.217354+0000","last_undegraded":"2024-03-16T10:07:42.217354+0000","last_fullsized":"2024-03-16T10:07:42.217354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:06.081507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963349+0000","last_change":"2024-03-16T10:06:08.346622+0000","last_active":"2024-03-16T10:07:13.963349+0000","last_peered":"2024-03-16T10:07:13.963349+0000","last_clean":"2024-03-16T10:07:13.963349+0000","last_became_active":"2024-03-16T10:06:08.346209+0000","last_became_peered":"2024-03-16T10:06:08.346209+0000","last_unstale":"2024-03-16T10:07:13.963349+0000","last_undegraded":"2024-03-16T10:07:13.963349+0000","last_fullsized":"2024-03-16T10:07:13.963349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:46:07.305218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924502+0000","last_change":"2024-03-16T10:06:03.227303+0000","last_active":"2024-03-16T10:07:13.924502+0000","last_peered":"2024-03-16T10:07:13.924502+0000","last_clean":"2024-03-16T10:07:13.924502+0000","last_became_active":"2024-03-16T10:06:03.226953+0000","last_became_peered":"2024-03-16T10:06:03.226953+0000","last_unstale":"2024-03-16T10:07:13.924502+0000","last_undegraded":"2024-03-16T10:07:13.924502+0000","last_fullsized":"2024-03-16T10:07:13.924502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:28:40.821171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211505+0000","last_change":"2024-03-16T10:06:04.220502+0000","last_active":"2024-03-16T10:07:42.211505+0000","last_peered":"2024-03-16T10:07:42.211505+0000","last_clean":"2024-03-16T10:07:42.211505+0000","last_became_active":"2024-03-16T10:06:04.220057+0000","last_became_peered":"2024-03-16T10:06:04.220057+0000","last_unstale":"2024-03-16T10:07:42.211505+0000","last_undegraded":"2024-03-16T10:07:42.211505+0000","last_fullsized":"2024-03-16T10:07:42.211505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:46:30.972382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963055+0000","last_change":"2024-03-16T10:06:08.346548+0000","last_active":"2024-03-16T10:07:13.963055+0000","last_peered":"2024-03-16T10:07:13.963055+0000","last_clean":"2024-03-16T10:07:13.963055+0000","last_became_active":"2024-03-16T10:06:08.346145+0000","last_became_peered":"2024-03-16T10:06:08.346145+0000","last_unstale":"2024-03-16T10:07:13.963055+0000","last_undegraded":"2024-03-16T10:07:13.963055+0000","last_fullsized":"2024-03-16T10:07:13.963055+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:51:49.221773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924325+0000","last_change":"2024-03-16T10:06:03.222623+0000","last_active":"2024-03-16T10:07:13.924325+0000","last_peered":"2024-03-16T10:07:13.924325+0000","last_clean":"2024-03-16T10:07:13.924325+0000","last_became_active":"2024-03-16T10:06:03.222300+0000","last_became_peered":"2024-03-16T10:06:03.222300+0000","last_unstale":"2024-03-16T10:07:13.924325+0000","last_undegraded":"2024-03-16T10:07:13.924325+0000","last_fullsized":"2024-03-16T10:07:13.924325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:49.397327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.200074+0000","last_change":"2024-03-16T10:07:14.202077+0000","last_active":"2024-03-16T10:07:42.200074+0000","last_peered":"2024-03-16T10:07:42.200074+0000","last_clean":"2024-03-16T10:07:42.200074+0000","last_became_active":"2024-03-16T10:07:14.201647+0000","last_became_peered":"2024-03-16T10:07:14.201647+0000","last_unstale":"2024-03-16T10:07:42.200074+0000","last_undegraded":"2024-03-16T10:07:42.200074+0000","last_fullsized":"2024-03-16T10:07:42.200074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:30.698520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,6,1],"acting":[9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926663+0000","last_change":"2024-03-16T10:06:08.330170+0000","last_active":"2024-03-16T10:07:13.926663+0000","last_peered":"2024-03-16T10:07:13.926663+0000","last_clean":"2024-03-16T10:07:13.926663+0000","last_became_active":"2024-03-16T10:06:08.329699+0000","last_became_peered":"2024-03-16T10:06:08.329699+0000","last_unstale":"2024-03-16T10:07:13.926663+0000","last_undegraded":"2024-03-16T10:07:13.926663+0000","last_fullsized":"2024-03-16T10:07:13.926663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:50.571845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.679684+0000","last_change":"2024-03-16T10:06:03.228039+0000","last_active":"2024-03-16T10:08:00.679684+0000","last_peered":"2024-03-16T10:08:00.679684+0000","last_clean":"2024-03-16T10:08:00.679684+0000","last_became_active":"2024-03-16T10:06:03.227715+0000","last_became_peered":"2024-03-16T10:06:03.227715+0000","last_unstale":"2024-03-16T10:08:00.679684+0000","last_undegraded":"2024-03-16T10:08:00.679684+0000","last_fullsized":"2024-03-16T10:08:00.679684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:13:08.980781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.188434+0000","last_change":"2024-03-16T10:06:04.214274+0000","last_active":"2024-03-16T10:07:42.188434+0000","last_peered":"2024-03-16T10:07:42.188434+0000","last_clean":"2024-03-16T10:07:42.188434+0000","last_became_active":"2024-03-16T10:06:04.213979+0000","last_became_peered":"2024-03-16T10:06:04.213979+0000","last_unstale":"2024-03-16T10:07:42.188434+0000","last_undegraded":"2024-03-16T10:07:42.188434+0000","last_fullsized":"2024-03-16T10:07:42.188434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:12.684609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927412+0000","last_change":"2024-03-16T10:06:08.346874+0000","last_active":"2024-03-16T10:07:13.927412+0000","last_peered":"2024-03-16T10:07:13.927412+0000","last_clean":"2024-03-16T10:07:13.927412+0000","last_became_active":"2024-03-16T10:06:08.346450+0000","last_became_peered":"2024-03-16T10:06:08.346450+0000","last_unstale":"2024-03-16T10:07:13.927412+0000","last_undegraded":"2024-03-16T10:07:13.927412+0000","last_fullsized":"2024-03-16T10:07:13.927412+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:09:12.159483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196008+0000","last_change":"2024-03-16T10:06:03.236088+0000","last_active":"2024-03-16T10:07:14.196008+0000","last_peered":"2024-03-16T10:07:14.196008+0000","last_clean":"2024-03-16T10:07:14.196008+0000","last_became_active":"2024-03-16T10:06:03.235768+0000","last_became_peered":"2024-03-16T10:06:03.235768+0000","last_unstale":"2024-03-16T10:07:14.196008+0000","last_undegraded":"2024-03-16T10:07:14.196008+0000","last_fullsized":"2024-03-16T10:07:14.196008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:57:38.813913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.207966+0000","last_change":"2024-03-16T10:06:04.224331+0000","last_active":"2024-03-16T10:07:42.207966+0000","last_peered":"2024-03-16T10:07:42.207966+0000","last_clean":"2024-03-16T10:07:42.207966+0000","last_became_active":"2024-03-16T10:06:04.223946+0000","last_became_peered":"2024-03-16T10:06:04.223946+0000","last_unstale":"2024-03-16T10:07:42.207966+0000","last_undegraded":"2024-03-16T10:07:42.207966+0000","last_fullsized":"2024-03-16T10:07:42.207966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:58:56.373290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927251+0000","last_change":"2024-03-16T10:06:08.344757+0000","last_active":"2024-03-16T10:07:13.927251+0000","last_peered":"2024-03-16T10:07:13.927251+0000","last_clean":"2024-03-16T10:07:13.927251+0000","last_became_active":"2024-03-16T10:06:08.344275+0000","last_became_peered":"2024-03-16T10:06:08.344275+0000","last_unstale":"2024-03-16T10:07:13.927251+0000","last_undegraded":"2024-03-16T10:07:13.927251+0000","last_fullsized":"2024-03-16T10:07:13.927251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:31:18.545734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.197197+0000","last_change":"2024-03-16T10:06:03.239619+0000","last_active":"2024-03-16T10:07:14.197197+0000","last_peered":"2024-03-16T10:07:14.197197+0000","last_clean":"2024-03-16T10:07:14.197197+0000","last_became_active":"2024-03-16T10:06:03.238982+0000","last_became_peered":"2024-03-16T10:06:03.238982+0000","last_unstale":"2024-03-16T10:07:14.197197+0000","last_undegraded":"2024-03-16T10:07:14.197197+0000","last_fullsized":"2024-03-16T10:07:14.197197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:20.787354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.213705+0000","last_change":"2024-03-16T10:06:04.221897+0000","last_active":"2024-03-16T10:07:42.213705+0000","last_peered":"2024-03-16T10:07:42.213705+0000","last_clean":"2024-03-16T10:07:42.213705+0000","last_became_active":"2024-03-16T10:06:04.218243+0000","last_became_peered":"2024-03-16T10:06:04.218243+0000","last_unstale":"2024-03-16T10:07:42.213705+0000","last_undegraded":"2024-03-16T10:07:42.213705+0000","last_fullsized":"2024-03-16T10:07:42.213705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:25.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.933682+0000","last_change":"2024-03-16T10:06:08.352722+0000","last_active":"2024-03-16T10:07:13.933682+0000","last_peered":"2024-03-16T10:07:13.933682+0000","last_clean":"2024-03-16T10:07:13.933682+0000","last_became_active":"2024-03-16T10:06:08.352292+0000","last_became_peered":"2024-03-16T10:06:08.352292+0000","last_unstale":"2024-03-16T10:07:13.933682+0000","last_undegraded":"2024-03-16T10:07:13.933682+0000","last_fullsized":"2024-03-16T10:07:13.933682+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:56:23.135158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195477+0000","last_change":"2024-03-16T10:06:03.224143+0000","last_active":"2024-03-16T10:07:14.195477+0000","last_peered":"2024-03-16T10:07:14.195477+0000","last_clean":"2024-03-16T10:07:14.195477+0000","last_became_active":"2024-03-16T10:06:03.223810+0000","last_became_peered":"2024-03-16T10:06:03.223810+0000","last_unstale":"2024-03-16T10:07:14.195477+0000","last_undegraded":"2024-03-16T10:07:14.195477+0000","last_fullsized":"2024-03-16T10:07:14.195477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:17:26.724292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212935+0000","last_change":"2024-03-16T10:06:04.232758+0000","last_active":"2024-03-16T10:07:42.212935+0000","last_peered":"2024-03-16T10:07:42.212935+0000","last_clean":"2024-03-16T10:07:42.212935+0000","last_became_active":"2024-03-16T10:06:04.232371+0000","last_became_peered":"2024-03-16T10:06:04.232371+0000","last_unstale":"2024-03-16T10:07:42.212935+0000","last_undegraded":"2024-03-16T10:07:42.212935+0000","last_fullsized":"2024-03-16T10:07:42.212935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:00.527708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.063109+0000","last_change":"2024-03-16T10:06:08.345360+0000","last_active":"2024-03-16T10:07:14.063109+0000","last_peered":"2024-03-16T10:07:14.063109+0000","last_clean":"2024-03-16T10:07:14.063109+0000","last_became_active":"2024-03-16T10:06:08.336245+0000","last_became_peered":"2024-03-16T10:06:08.336245+0000","last_unstale":"2024-03-16T10:07:14.063109+0000","last_undegraded":"2024-03-16T10:07:14.063109+0000","last_fullsized":"2024-03-16T10:07:14.063109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:14:33.324555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":195,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.723288+0000","last_change":"2024-03-16T10:06:03.237015+0000","last_active":"2024-03-16T10:08:00.723288+0000","last_peered":"2024-03-16T10:08:00.723288+0000","last_clean":"2024-03-16T10:08:00.723288+0000","last_became_active":"2024-03-16T10:06:03.236722+0000","last_became_peered":"2024-03-16T10:06:03.236722+0000","last_unstale":"2024-03-16T10:08:00.723288+0000","last_undegraded":"2024-03-16T10:08:00.723288+0000","last_fullsized":"2024-03-16T10:08:00.723288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:10:03.403854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210293+0000","last_change":"2024-03-16T10:06:04.209017+0000","last_active":"2024-03-16T10:07:42.210293+0000","last_peered":"2024-03-16T10:07:42.210293+0000","last_clean":"2024-03-16T10:07:42.210293+0000","last_became_active":"2024-03-16T10:06:04.208665+0000","last_became_peered":"2024-03-16T10:06:04.208665+0000","last_unstale":"2024-03-16T10:07:42.210293+0000","last_undegraded":"2024-03-16T10:07:42.210293+0000","last_fullsized":"2024-03-16T10:07:42.210293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:27.074480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928969+0000","last_change":"2024-03-16T10:06:08.340510+0000","last_active":"2024-03-16T10:07:13.928969+0000","last_peered":"2024-03-16T10:07:13.928969+0000","last_clean":"2024-03-16T10:07:13.928969+0000","last_became_active":"2024-03-16T10:06:08.340070+0000","last_became_peered":"2024-03-16T10:06:08.340070+0000","last_unstale":"2024-03-16T10:07:13.928969+0000","last_undegraded":"2024-03-16T10:07:13.928969+0000","last_fullsized":"2024-03-16T10:07:13.928969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:33:55.038855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926601+0000","last_change":"2024-03-16T10:06:03.225770+0000","last_active":"2024-03-16T10:07:13.926601+0000","last_peered":"2024-03-16T10:07:13.926601+0000","last_clean":"2024-03-16T10:07:13.926601+0000","last_became_active":"2024-03-16T10:06:03.223734+0000","last_became_peered":"2024-03-16T10:06:03.223734+0000","last_unstale":"2024-03-16T10:07:13.926601+0000","last_undegraded":"2024-03-16T10:07:13.926601+0000","last_fullsized":"2024-03-16T10:07:13.926601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:24:13.764147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212151+0000","last_change":"2024-03-16T10:07:14.197803+0000","last_active":"2024-03-16T10:07:42.212151+0000","last_peered":"2024-03-16T10:07:42.212151+0000","last_clean":"2024-03-16T10:07:42.212151+0000","last_became_active":"2024-03-16T10:07:14.197195+0000","last_became_peered":"2024-03-16T10:07:14.197195+0000","last_unstale":"2024-03-16T10:07:42.212151+0000","last_undegraded":"2024-03-16T10:07:42.212151+0000","last_fullsized":"2024-03-16T10:07:42.212151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:01:03.480157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963468+0000","last_change":"2024-03-16T10:06:08.345595+0000","last_active":"2024-03-16T10:07:13.963468+0000","last_peered":"2024-03-16T10:07:13.963468+0000","last_clean":"2024-03-16T10:07:13.963468+0000","last_became_active":"2024-03-16T10:06:08.345205+0000","last_became_peered":"2024-03-16T10:06:08.345205+0000","last_unstale":"2024-03-16T10:07:13.963468+0000","last_undegraded":"2024-03-16T10:07:13.963468+0000","last_fullsized":"2024-03-16T10:07:13.963468+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:37:51.067056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195692+0000","last_change":"2024-03-16T10:06:03.219716+0000","last_active":"2024-03-16T10:07:14.195692+0000","last_peered":"2024-03-16T10:07:14.195692+0000","last_clean":"2024-03-16T10:07:14.195692+0000","last_became_active":"2024-03-16T10:06:03.219375+0000","last_became_peered":"2024-03-16T10:06:03.219375+0000","last_unstale":"2024-03-16T10:07:14.195692+0000","last_undegraded":"2024-03-16T10:07:14.195692+0000","last_fullsized":"2024-03-16T10:07:14.195692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:23:51.038521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211478+0000","last_change":"2024-03-16T10:06:04.193605+0000","last_active":"2024-03-16T10:07:42.211478+0000","last_peered":"2024-03-16T10:07:42.211478+0000","last_clean":"2024-03-16T10:07:42.211478+0000","last_became_active":"2024-03-16T10:06:04.193295+0000","last_became_peered":"2024-03-16T10:06:04.193295+0000","last_unstale":"2024-03-16T10:07:42.211478+0000","last_undegraded":"2024-03-16T10:07:42.211478+0000","last_fullsized":"2024-03-16T10:07:42.211478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:10:04.140279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.931040+0000","last_change":"2024-03-16T10:06:08.338862+0000","last_active":"2024-03-16T10:07:13.931040+0000","last_peered":"2024-03-16T10:07:13.931040+0000","last_clean":"2024-03-16T10:07:13.931040+0000","last_became_active":"2024-03-16T10:06:08.337221+0000","last_became_peered":"2024-03-16T10:06:08.337221+0000","last_unstale":"2024-03-16T10:07:13.931040+0000","last_undegraded":"2024-03-16T10:07:13.931040+0000","last_fullsized":"2024-03-16T10:07:13.931040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:28:51.251299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":235,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:01.683265+0000","last_change":"2024-03-16T10:06:03.220555+0000","last_active":"2024-03-16T10:08:01.683265+0000","last_peered":"2024-03-16T10:08:01.683265+0000","last_clean":"2024-03-16T10:08:01.683265+0000","last_became_active":"2024-03-16T10:06:03.220224+0000","last_became_peered":"2024-03-16T10:06:03.220224+0000","last_unstale":"2024-03-16T10:08:01.683265+0000","last_undegraded":"2024-03-16T10:08:01.683265+0000","last_fullsized":"2024-03-16T10:08:01.683265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:12:47.568132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62874,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":171,"num_read_kb":147,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.191212+0000","last_change":"2024-03-16T10:06:04.195957+0000","last_active":"2024-03-16T10:07:42.191212+0000","last_peered":"2024-03-16T10:07:42.191212+0000","last_clean":"2024-03-16T10:07:42.191212+0000","last_became_active":"2024-03-16T10:06:04.195539+0000","last_became_peered":"2024-03-16T10:06:04.195539+0000","last_unstale":"2024-03-16T10:07:42.191212+0000","last_undegraded":"2024-03-16T10:07:42.191212+0000","last_fullsized":"2024-03-16T10:07:42.191212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:39:45.035936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196250+0000","last_change":"2024-03-16T10:06:08.347941+0000","last_active":"2024-03-16T10:07:14.196250+0000","last_peered":"2024-03-16T10:07:14.196250+0000","last_clean":"2024-03-16T10:07:14.196250+0000","last_became_active":"2024-03-16T10:06:08.347517+0000","last_became_peered":"2024-03-16T10:06:08.347517+0000","last_unstale":"2024-03-16T10:07:14.196250+0000","last_undegraded":"2024-03-16T10:07:14.196250+0000","last_fullsized":"2024-03-16T10:07:14.196250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:51:08.696206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927465+0000","last_change":"2024-03-16T10:06:03.235652+0000","last_active":"2024-03-16T10:07:13.927465+0000","last_peered":"2024-03-16T10:07:13.927465+0000","last_clean":"2024-03-16T10:07:13.927465+0000","last_became_active":"2024-03-16T10:06:03.235219+0000","last_became_peered":"2024-03-16T10:06:03.235219+0000","last_unstale":"2024-03-16T10:07:13.927465+0000","last_undegraded":"2024-03-16T10:07:13.927465+0000","last_fullsized":"2024-03-16T10:07:13.927465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:23.691695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214350+0000","last_change":"2024-03-16T10:06:04.225460+0000","last_active":"2024-03-16T10:07:42.214350+0000","last_peered":"2024-03-16T10:07:42.214350+0000","last_clean":"2024-03-16T10:07:42.214350+0000","last_became_active":"2024-03-16T10:06:04.225019+0000","last_became_peered":"2024-03-16T10:06:04.225019+0000","last_unstale":"2024-03-16T10:07:42.214350+0000","last_undegraded":"2024-03-16T10:07:42.214350+0000","last_fullsized":"2024-03-16T10:07:42.214350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:42:06.525298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.925840+0000","last_change":"2024-03-16T10:06:08.344775+0000","last_active":"2024-03-16T10:07:13.925840+0000","last_peered":"2024-03-16T10:07:13.925840+0000","last_clean":"2024-03-16T10:07:13.925840+0000","last_became_active":"2024-03-16T10:06:08.344360+0000","last_became_peered":"2024-03-16T10:06:08.344360+0000","last_unstale":"2024-03-16T10:07:13.925840+0000","last_undegraded":"2024-03-16T10:07:13.925840+0000","last_fullsized":"2024-03-16T10:07:13.925840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:26.710955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195992+0000","last_change":"2024-03-16T10:06:03.236887+0000","last_active":"2024-03-16T10:07:14.195992+0000","last_peered":"2024-03-16T10:07:14.195992+0000","last_clean":"2024-03-16T10:07:14.195992+0000","last_became_active":"2024-03-16T10:06:03.236454+0000","last_became_peered":"2024-03-16T10:06:03.236454+0000","last_unstale":"2024-03-16T10:07:14.195992+0000","last_undegraded":"2024-03-16T10:07:14.195992+0000","last_fullsized":"2024-03-16T10:07:14.195992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:59:03.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.203957+0000","last_change":"2024-03-16T10:06:04.220008+0000","last_active":"2024-03-16T10:07:42.203957+0000","last_peered":"2024-03-16T10:07:42.203957+0000","last_clean":"2024-03-16T10:07:42.203957+0000","last_became_active":"2024-03-16T10:06:04.219683+0000","last_became_peered":"2024-03-16T10:06:04.219683+0000","last_unstale":"2024-03-16T10:07:42.203957+0000","last_undegraded":"2024-03-16T10:07:42.203957+0000","last_fullsized":"2024-03-16T10:07:42.203957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:34:35.577474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.930260+0000","last_change":"2024-03-16T10:06:08.352793+0000","last_active":"2024-03-16T10:07:13.930260+0000","last_peered":"2024-03-16T10:07:13.930260+0000","last_clean":"2024-03-16T10:07:13.930260+0000","last_became_active":"2024-03-16T10:06:08.352272+0000","last_became_peered":"2024-03-16T10:06:08.352272+0000","last_unstale":"2024-03-16T10:07:13.930260+0000","last_undegraded":"2024-03-16T10:07:13.930260+0000","last_fullsized":"2024-03-16T10:07:13.930260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:53:39.835078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":121,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:02.757938+0000","last_change":"2024-03-16T10:06:03.224301+0000","last_active":"2024-03-16T10:08:02.757938+0000","last_peered":"2024-03-16T10:08:02.757938+0000","last_clean":"2024-03-16T10:08:02.757938+0000","last_became_active":"2024-03-16T10:06:03.223899+0000","last_became_peered":"2024-03-16T10:06:03.223899+0000","last_unstale":"2024-03-16T10:08:02.757938+0000","last_undegraded":"2024-03-16T10:08:02.757938+0000","last_fullsized":"2024-03-16T10:08:02.757938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:39.244106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":88,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.198542+0000","last_change":"2024-03-16T10:06:04.193649+0000","last_active":"2024-03-16T10:07:42.198542+0000","last_peered":"2024-03-16T10:07:42.198542+0000","last_clean":"2024-03-16T10:07:42.198542+0000","last_became_active":"2024-03-16T10:06:04.193275+0000","last_became_peered":"2024-03-16T10:06:04.193275+0000","last_unstale":"2024-03-16T10:07:42.198542+0000","last_undegraded":"2024-03-16T10:07:42.198542+0000","last_fullsized":"2024-03-16T10:07:42.198542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:59.597769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198270+0000","last_change":"2024-03-16T10:06:08.349918+0000","last_active":"2024-03-16T10:07:14.198270+0000","last_peered":"2024-03-16T10:07:14.198270+0000","last_clean":"2024-03-16T10:07:14.198270+0000","last_became_active":"2024-03-16T10:06:08.349506+0000","last_became_peered":"2024-03-16T10:06:08.349506+0000","last_unstale":"2024-03-16T10:07:14.198270+0000","last_undegraded":"2024-03-16T10:07:14.198270+0000","last_fullsized":"2024-03-16T10:07:14.198270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:05:08.146594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.946883+0000","last_change":"2024-03-16T10:07:13.946883+0000","last_active":"2024-03-16T10:07:13.946883+0000","last_peered":"2024-03-16T10:07:13.946883+0000","last_clean":"2024-03-16T10:07:13.946883+0000","last_became_active":"2024-03-16T10:07:13.934745+0000","last_became_peered":"2024-03-16T10:07:13.934745+0000","last_unstale":"2024-03-16T10:07:13.946883+0000","last_undegraded":"2024-03-16T10:07:13.946883+0000","last_fullsized":"2024-03-16T10:07:13.946883+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:26:42.936333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.201757+0000","last_change":"2024-03-16T10:06:04.194629+0000","last_active":"2024-03-16T10:07:42.201757+0000","last_peered":"2024-03-16T10:07:42.201757+0000","last_clean":"2024-03-16T10:07:42.201757+0000","last_became_active":"2024-03-16T10:06:04.194306+0000","last_became_peered":"2024-03-16T10:06:04.194306+0000","last_unstale":"2024-03-16T10:07:42.201757+0000","last_undegraded":"2024-03-16T10:07:42.201757+0000","last_fullsized":"2024-03-16T10:07:42.201757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:36.455569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195547+0000","last_change":"2024-03-16T10:06:08.349269+0000","last_active":"2024-03-16T10:07:14.195547+0000","last_peered":"2024-03-16T10:07:14.195547+0000","last_clean":"2024-03-16T10:07:14.195547+0000","last_became_active":"2024-03-16T10:06:08.348824+0000","last_became_peered":"2024-03-16T10:06:08.348824+0000","last_unstale":"2024-03-16T10:07:14.195547+0000","last_undegraded":"2024-03-16T10:07:14.195547+0000","last_fullsized":"2024-03-16T10:07:14.195547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:26.560781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924724+0000","last_change":"2024-03-16T10:06:03.221066+0000","last_active":"2024-03-16T10:07:13.924724+0000","last_peered":"2024-03-16T10:07:13.924724+0000","last_clean":"2024-03-16T10:07:13.924724+0000","last_became_active":"2024-03-16T10:06:03.219908+0000","last_became_peered":"2024-03-16T10:06:03.219908+0000","last_unstale":"2024-03-16T10:07:13.924724+0000","last_undegraded":"2024-03-16T10:07:13.924724+0000","last_fullsized":"2024-03-16T10:07:13.924724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:41.026293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.191891+0000","last_change":"2024-03-16T10:07:14.199329+0000","last_active":"2024-03-16T10:07:42.191891+0000","last_peered":"2024-03-16T10:07:42.191891+0000","last_clean":"2024-03-16T10:07:42.191891+0000","last_became_active":"2024-03-16T10:07:14.198920+0000","last_became_peered":"2024-03-16T10:07:14.198920+0000","last_unstale":"2024-03-16T10:07:42.191891+0000","last_undegraded":"2024-03-16T10:07:42.191891+0000","last_fullsized":"2024-03-16T10:07:42.191891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:06:32.157083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924734+0000","last_change":"2024-03-16T10:06:08.529441+0000","last_active":"2024-03-16T10:07:13.924734+0000","last_peered":"2024-03-16T10:07:13.924734+0000","last_clean":"2024-03-16T10:07:13.924734+0000","last_became_active":"2024-03-16T10:06:08.528801+0000","last_became_peered":"2024-03-16T10:06:08.528801+0000","last_unstale":"2024-03-16T10:07:13.924734+0000","last_undegraded":"2024-03-16T10:07:13.924734+0000","last_fullsized":"2024-03-16T10:07:13.924734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:23:50.592415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.929322+0000","last_change":"2024-03-16T10:06:03.236260+0000","last_active":"2024-03-16T10:07:13.929322+0000","last_peered":"2024-03-16T10:07:13.929322+0000","last_clean":"2024-03-16T10:07:13.929322+0000","last_became_active":"2024-03-16T10:06:03.235792+0000","last_became_peered":"2024-03-16T10:06:03.235792+0000","last_unstale":"2024-03-16T10:07:13.929322+0000","last_undegraded":"2024-03-16T10:07:13.929322+0000","last_fullsized":"2024-03-16T10:07:13.929322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:11:59.364244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210511+0000","last_change":"2024-03-16T10:06:04.226962+0000","last_active":"2024-03-16T10:07:42.210511+0000","last_peered":"2024-03-16T10:07:42.210511+0000","last_clean":"2024-03-16T10:07:42.210511+0000","last_became_active":"2024-03-16T10:06:04.226668+0000","last_became_peered":"2024-03-16T10:06:04.226668+0000","last_unstale":"2024-03-16T10:07:42.210511+0000","last_undegraded":"2024-03-16T10:07:42.210511+0000","last_fullsized":"2024-03-16T10:07:42.210511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:53.097834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198018+0000","last_change":"2024-03-16T10:06:08.343528+0000","last_active":"2024-03-16T10:07:14.198018+0000","last_peered":"2024-03-16T10:07:14.198018+0000","last_clean":"2024-03-16T10:07:14.198018+0000","last_became_active":"2024-03-16T10:06:08.340969+0000","last_became_peered":"2024-03-16T10:06:08.340969+0000","last_unstale":"2024-03-16T10:07:14.198018+0000","last_undegraded":"2024-03-16T10:07:14.198018+0000","last_fullsized":"2024-03-16T10:07:14.198018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:54:33.028756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.717346+0000","last_change":"2024-03-16T10:06:03.236166+0000","last_active":"2024-03-16T10:08:00.717346+0000","last_peered":"2024-03-16T10:08:00.717346+0000","last_clean":"2024-03-16T10:08:00.717346+0000","last_became_active":"2024-03-16T10:06:03.235825+0000","last_became_peered":"2024-03-16T10:06:03.235825+0000","last_unstale":"2024-03-16T10:08:00.717346+0000","last_undegraded":"2024-03-16T10:08:00.717346+0000","last_fullsized":"2024-03-16T10:08:00.717346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:39:34.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.208431+0000","last_change":"2024-03-16T10:06:04.220002+0000","last_active":"2024-03-16T10:07:42.208431+0000","last_peered":"2024-03-16T10:07:42.208431+0000","last_clean":"2024-03-16T10:07:42.208431+0000","last_became_active":"2024-03-16T10:06:04.219623+0000","last_became_peered":"2024-03-16T10:06:04.219623+0000","last_unstale":"2024-03-16T10:07:42.208431+0000","last_undegraded":"2024-03-16T10:07:42.208431+0000","last_fullsized":"2024-03-16T10:07:42.208431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:39:35.708301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.197936+0000","last_change":"2024-03-16T10:06:08.530396+0000","last_active":"2024-03-16T10:07:14.197936+0000","last_peered":"2024-03-16T10:07:14.197936+0000","last_clean":"2024-03-16T10:07:14.197936+0000","last_became_active":"2024-03-16T10:06:08.529102+0000","last_became_peered":"2024-03-16T10:06:08.529102+0000","last_unstale":"2024-03-16T10:07:14.197936+0000","last_undegraded":"2024-03-16T10:07:14.197936+0000","last_fullsized":"2024-03-16T10:07:14.197936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:36.408884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.194154+0000","last_change":"2024-03-16T10:07:13.964902+0000","last_active":"2024-03-16T10:07:42.194154+0000","last_peered":"2024-03-16T10:07:42.194154+0000","last_clean":"2024-03-16T10:07:42.194154+0000","last_became_active":"2024-03-16T10:07:13.963970+0000","last_became_peered":"2024-03-16T10:07:13.963970+0000","last_unstale":"2024-03-16T10:07:42.194154+0000","last_undegraded":"2024-03-16T10:07:42.194154+0000","last_fullsized":"2024-03-16T10:07:42.194154+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:15.626355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:00.753374+0000","last_change":"2024-03-16T10:06:03.247109+0000","last_active":"2024-03-16T10:08:00.753374+0000","last_peered":"2024-03-16T10:08:00.753374+0000","last_clean":"2024-03-16T10:08:00.753374+0000","last_became_active":"2024-03-16T10:06:03.246515+0000","last_became_peered":"2024-03-16T10:06:03.246515+0000","last_unstale":"2024-03-16T10:08:00.753374+0000","last_undegraded":"2024-03-16T10:08:00.753374+0000","last_fullsized":"2024-03-16T10:08:00.753374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:29.744828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926737+0000","last_change":"2024-03-16T10:06:08.334587+0000","last_active":"2024-03-16T10:07:13.926737+0000","last_peered":"2024-03-16T10:07:13.926737+0000","last_clean":"2024-03-16T10:07:13.926737+0000","last_became_active":"2024-03-16T10:06:08.334248+0000","last_became_peered":"2024-03-16T10:06:08.334248+0000","last_unstale":"2024-03-16T10:07:13.926737+0000","last_undegraded":"2024-03-16T10:07:13.926737+0000","last_fullsized":"2024-03-16T10:07:13.926737+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:33:28.909540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.212641+0000","last_change":"2024-03-16T10:06:04.221069+0000","last_active":"2024-03-16T10:07:42.212641+0000","last_peered":"2024-03-16T10:07:42.212641+0000","last_clean":"2024-03-16T10:07:42.212641+0000","last_became_active":"2024-03-16T10:06:04.218197+0000","last_became_peered":"2024-03-16T10:06:04.218197+0000","last_unstale":"2024-03-16T10:07:42.212641+0000","last_undegraded":"2024-03-16T10:07:42.212641+0000","last_fullsized":"2024-03-16T10:07:42.212641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:38:25.262182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963402+0000","last_change":"2024-03-16T10:06:03.221284+0000","last_active":"2024-03-16T10:07:13.963402+0000","last_peered":"2024-03-16T10:07:13.963402+0000","last_clean":"2024-03-16T10:07:13.963402+0000","last_became_active":"2024-03-16T10:06:03.220467+0000","last_became_peered":"2024-03-16T10:06:03.220467+0000","last_unstale":"2024-03-16T10:07:13.963402+0000","last_undegraded":"2024-03-16T10:07:13.963402+0000","last_fullsized":"2024-03-16T10:07:13.963402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:31:19.753996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":359,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926333+0000","last_change":"2024-03-16T10:04:45.615656+0000","last_active":"2024-03-16T10:07:13.926333+0000","last_peered":"2024-03-16T10:07:13.926333+0000","last_clean":"2024-03-16T10:07:13.926333+0000","last_became_active":"2024-03-16T10:04:45.615261+0000","last_became_peered":"2024-03-16T10:04:45.615261+0000","last_unstale":"2024-03-16T10:07:13.926333+0000","last_undegraded":"2024-03-16T10:07:13.926333+0000","last_fullsized":"2024-03-16T10:07:13.926333+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:00:56.880115+0000","last_clean_scrub_stamp":"2024-03-16T10:00:56.880115+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:06.546840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196230+0000","last_change":"2024-03-16T10:06:08.529863+0000","last_active":"2024-03-16T10:07:14.196230+0000","last_peered":"2024-03-16T10:07:14.196230+0000","last_clean":"2024-03-16T10:07:14.196230+0000","last_became_active":"2024-03-16T10:06:08.528934+0000","last_became_peered":"2024-03-16T10:06:08.528934+0000","last_unstale":"2024-03-16T10:07:14.196230+0000","last_undegraded":"2024-03-16T10:07:14.196230+0000","last_fullsized":"2024-03-16T10:07:14.196230+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:35:40.443983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193428+0000","last_change":"2024-03-16T10:06:04.206776+0000","last_active":"2024-03-16T10:07:42.193428+0000","last_peered":"2024-03-16T10:07:42.193428+0000","last_clean":"2024-03-16T10:07:42.193428+0000","last_became_active":"2024-03-16T10:06:04.206333+0000","last_became_peered":"2024-03-16T10:06:04.206333+0000","last_unstale":"2024-03-16T10:07:42.193428+0000","last_undegraded":"2024-03-16T10:07:42.193428+0000","last_fullsized":"2024-03-16T10:07:42.193428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:00:34.634986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926410+0000","last_change":"2024-03-16T10:06:03.226921+0000","last_active":"2024-03-16T10:07:13.926410+0000","last_peered":"2024-03-16T10:07:13.926410+0000","last_clean":"2024-03-16T10:07:13.926410+0000","last_became_active":"2024-03-16T10:06:03.226596+0000","last_became_peered":"2024-03-16T10:06:03.226596+0000","last_unstale":"2024-03-16T10:07:13.926410+0000","last_undegraded":"2024-03-16T10:07:13.926410+0000","last_fullsized":"2024-03-16T10:07:13.926410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:03:50.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.925983+0000","last_change":"2024-03-16T10:06:08.344432+0000","last_active":"2024-03-16T10:07:13.925983+0000","last_peered":"2024-03-16T10:07:13.925983+0000","last_clean":"2024-03-16T10:07:13.925983+0000","last_became_active":"2024-03-16T10:06:08.343983+0000","last_became_peered":"2024-03-16T10:06:08.343983+0000","last_unstale":"2024-03-16T10:07:13.925983+0000","last_undegraded":"2024-03-16T10:07:13.925983+0000","last_fullsized":"2024-03-16T10:07:13.925983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:41.686700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.924317+0000","last_change":"2024-03-16T10:06:03.232806+0000","last_active":"2024-03-16T10:07:13.924317+0000","last_peered":"2024-03-16T10:07:13.924317+0000","last_clean":"2024-03-16T10:07:13.924317+0000","last_became_active":"2024-03-16T10:06:03.232465+0000","last_became_peered":"2024-03-16T10:06:03.232465+0000","last_unstale":"2024-03-16T10:07:13.924317+0000","last_undegraded":"2024-03-16T10:07:13.924317+0000","last_fullsized":"2024-03-16T10:07:13.924317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:36.927753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.200103+0000","last_change":"2024-03-16T10:06:04.215899+0000","last_active":"2024-03-16T10:07:42.200103+0000","last_peered":"2024-03-16T10:07:42.200103+0000","last_clean":"2024-03-16T10:07:42.200103+0000","last_became_active":"2024-03-16T10:06:04.215474+0000","last_became_peered":"2024-03-16T10:06:04.215474+0000","last_unstale":"2024-03-16T10:07:42.200103+0000","last_undegraded":"2024-03-16T10:07:42.200103+0000","last_fullsized":"2024-03-16T10:07:42.200103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:27:26.663702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.062701+0000","last_change":"2024-03-16T10:06:08.530852+0000","last_active":"2024-03-16T10:07:14.062701+0000","last_peered":"2024-03-16T10:07:14.062701+0000","last_clean":"2024-03-16T10:07:14.062701+0000","last_became_active":"2024-03-16T10:06:08.529386+0000","last_became_peered":"2024-03-16T10:06:08.529386+0000","last_unstale":"2024-03-16T10:07:14.062701+0000","last_undegraded":"2024-03-16T10:07:14.062701+0000","last_fullsized":"2024-03-16T10:07:14.062701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:37:17.201323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.929540+0000","last_change":"2024-03-16T10:06:03.244012+0000","last_active":"2024-03-16T10:07:13.929540+0000","last_peered":"2024-03-16T10:07:13.929540+0000","last_clean":"2024-03-16T10:07:13.929540+0000","last_became_active":"2024-03-16T10:06:03.243612+0000","last_became_peered":"2024-03-16T10:06:03.243612+0000","last_unstale":"2024-03-16T10:07:13.929540+0000","last_undegraded":"2024-03-16T10:07:13.929540+0000","last_fullsized":"2024-03-16T10:07:13.929540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:32:34.807111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.208446+0000","last_change":"2024-03-16T10:06:04.226678+0000","last_active":"2024-03-16T10:07:42.208446+0000","last_peered":"2024-03-16T10:07:42.208446+0000","last_clean":"2024-03-16T10:07:42.208446+0000","last_became_active":"2024-03-16T10:06:04.226214+0000","last_became_peered":"2024-03-16T10:06:04.226214+0000","last_unstale":"2024-03-16T10:07:42.208446+0000","last_undegraded":"2024-03-16T10:07:42.208446+0000","last_fullsized":"2024-03-16T10:07:42.208446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:49:05.732310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.927156+0000","last_change":"2024-03-16T10:06:08.326985+0000","last_active":"2024-03-16T10:07:13.927156+0000","last_peered":"2024-03-16T10:07:13.927156+0000","last_clean":"2024-03-16T10:07:13.927156+0000","last_became_active":"2024-03-16T10:06:08.326429+0000","last_became_peered":"2024-03-16T10:06:08.326429+0000","last_unstale":"2024-03-16T10:07:13.927156+0000","last_undegraded":"2024-03-16T10:07:13.927156+0000","last_fullsized":"2024-03-16T10:07:13.927156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:44:59.855608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963582+0000","last_change":"2024-03-16T10:06:03.232910+0000","last_active":"2024-03-16T10:07:13.963582+0000","last_peered":"2024-03-16T10:07:13.963582+0000","last_clean":"2024-03-16T10:07:13.963582+0000","last_became_active":"2024-03-16T10:06:03.232682+0000","last_became_peered":"2024-03-16T10:06:03.232682+0000","last_unstale":"2024-03-16T10:07:13.963582+0000","last_undegraded":"2024-03-16T10:07:13.963582+0000","last_fullsized":"2024-03-16T10:07:13.963582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:32:56.731620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.215888+0000","last_change":"2024-03-16T10:06:04.227189+0000","last_active":"2024-03-16T10:07:42.215888+0000","last_peered":"2024-03-16T10:07:42.215888+0000","last_clean":"2024-03-16T10:07:42.215888+0000","last_became_active":"2024-03-16T10:06:04.226723+0000","last_became_peered":"2024-03-16T10:06:04.226723+0000","last_unstale":"2024-03-16T10:07:42.215888+0000","last_undegraded":"2024-03-16T10:07:42.215888+0000","last_fullsized":"2024-03-16T10:07:42.215888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:47.873938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963191+0000","last_change":"2024-03-16T10:06:08.529316+0000","last_active":"2024-03-16T10:07:13.963191+0000","last_peered":"2024-03-16T10:07:13.963191+0000","last_clean":"2024-03-16T10:07:13.963191+0000","last_became_active":"2024-03-16T10:06:08.528903+0000","last_became_peered":"2024-03-16T10:06:08.528903+0000","last_unstale":"2024-03-16T10:07:13.963191+0000","last_undegraded":"2024-03-16T10:07:13.963191+0000","last_fullsized":"2024-03-16T10:07:13.963191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:19:33.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928101+0000","last_change":"2024-03-16T10:06:03.233819+0000","last_active":"2024-03-16T10:07:13.928101+0000","last_peered":"2024-03-16T10:07:13.928101+0000","last_clean":"2024-03-16T10:07:13.928101+0000","last_became_active":"2024-03-16T10:06:03.233593+0000","last_became_peered":"2024-03-16T10:06:03.233593+0000","last_unstale":"2024-03-16T10:07:13.928101+0000","last_undegraded":"2024-03-16T10:07:13.928101+0000","last_fullsized":"2024-03-16T10:07:13.928101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:18.847248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.165905+0000","last_change":"2024-03-16T10:06:04.192682+0000","last_active":"2024-03-16T10:07:42.165905+0000","last_peered":"2024-03-16T10:07:42.165905+0000","last_clean":"2024-03-16T10:07:42.165905+0000","last_became_active":"2024-03-16T10:06:04.192305+0000","last_became_peered":"2024-03-16T10:06:04.192305+0000","last_unstale":"2024-03-16T10:07:42.165905+0000","last_undegraded":"2024-03-16T10:07:42.165905+0000","last_fullsized":"2024-03-16T10:07:42.165905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:31:46.718663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.930770+0000","last_change":"2024-03-16T10:06:08.355342+0000","last_active":"2024-03-16T10:07:13.930770+0000","last_peered":"2024-03-16T10:07:13.930770+0000","last_clean":"2024-03-16T10:07:13.930770+0000","last_became_active":"2024-03-16T10:06:08.351547+0000","last_became_peered":"2024-03-16T10:06:08.351547+0000","last_unstale":"2024-03-16T10:07:13.930770+0000","last_undegraded":"2024-03-16T10:07:13.930770+0000","last_fullsized":"2024-03-16T10:07:13.930770+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:01:00.777949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.062906+0000","last_change":"2024-03-16T10:06:03.242248+0000","last_active":"2024-03-16T10:07:14.062906+0000","last_peered":"2024-03-16T10:07:14.062906+0000","last_clean":"2024-03-16T10:07:14.062906+0000","last_became_active":"2024-03-16T10:06:03.241842+0000","last_became_peered":"2024-03-16T10:06:03.241842+0000","last_unstale":"2024-03-16T10:07:14.062906+0000","last_undegraded":"2024-03-16T10:07:14.062906+0000","last_fullsized":"2024-03-16T10:07:14.062906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:03:35.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193391+0000","last_change":"2024-03-16T10:06:04.221790+0000","last_active":"2024-03-16T10:07:42.193391+0000","last_peered":"2024-03-16T10:07:42.193391+0000","last_clean":"2024-03-16T10:07:42.193391+0000","last_became_active":"2024-03-16T10:06:04.221475+0000","last_became_peered":"2024-03-16T10:06:04.221475+0000","last_unstale":"2024-03-16T10:07:42.193391+0000","last_undegraded":"2024-03-16T10:07:42.193391+0000","last_fullsized":"2024-03-16T10:07:42.193391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:05:50.502042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963254+0000","last_change":"2024-03-16T10:06:08.337906+0000","last_active":"2024-03-16T10:07:13.963254+0000","last_peered":"2024-03-16T10:07:13.963254+0000","last_clean":"2024-03-16T10:07:13.963254+0000","last_became_active":"2024-03-16T10:06:08.337444+0000","last_became_peered":"2024-03-16T10:06:08.337444+0000","last_unstale":"2024-03-16T10:07:13.963254+0000","last_undegraded":"2024-03-16T10:07:13.963254+0000","last_fullsized":"2024-03-16T10:07:13.963254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:09:03.997606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.926632+0000","last_change":"2024-03-16T10:06:03.235717+0000","last_active":"2024-03-16T10:07:13.926632+0000","last_peered":"2024-03-16T10:07:13.926632+0000","last_clean":"2024-03-16T10:07:13.926632+0000","last_became_active":"2024-03-16T10:06:03.235331+0000","last_became_peered":"2024-03-16T10:06:03.235331+0000","last_unstale":"2024-03-16T10:07:13.926632+0000","last_undegraded":"2024-03-16T10:07:13.926632+0000","last_fullsized":"2024-03-16T10:07:13.926632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:22:08.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214287+0000","last_change":"2024-03-16T10:06:04.229219+0000","last_active":"2024-03-16T10:07:42.214287+0000","last_peered":"2024-03-16T10:07:42.214287+0000","last_clean":"2024-03-16T10:07:42.214287+0000","last_became_active":"2024-03-16T10:06:04.226850+0000","last_became_peered":"2024-03-16T10:06:04.226850+0000","last_unstale":"2024-03-16T10:07:42.214287+0000","last_undegraded":"2024-03-16T10:07:42.214287+0000","last_fullsized":"2024-03-16T10:07:42.214287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:22:48.920858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.963311+0000","last_change":"2024-03-16T10:06:08.529201+0000","last_active":"2024-03-16T10:07:13.963311+0000","last_peered":"2024-03-16T10:07:13.963311+0000","last_clean":"2024-03-16T10:07:13.963311+0000","last_became_active":"2024-03-16T10:06:08.528782+0000","last_became_peered":"2024-03-16T10:06:08.528782+0000","last_unstale":"2024-03-16T10:07:13.963311+0000","last_undegraded":"2024-03-16T10:07:13.963311+0000","last_fullsized":"2024-03-16T10:07:13.963311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:35:38.267168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.968463+0000","last_change":"2024-03-16T10:07:13.968463+0000","last_active":"2024-03-16T10:07:13.968463+0000","last_peered":"2024-03-16T10:07:13.968463+0000","last_clean":"2024-03-16T10:07:13.968463+0000","last_became_active":"2024-03-16T10:07:13.964144+0000","last_became_peered":"2024-03-16T10:07:13.964144+0000","last_unstale":"2024-03-16T10:07:13.968463+0000","last_undegraded":"2024-03-16T10:07:13.968463+0000","last_fullsized":"2024-03-16T10:07:13.968463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:30:16.852944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.210780+0000","last_change":"2024-03-16T10:06:04.217699+0000","last_active":"2024-03-16T10:07:42.210780+0000","last_peered":"2024-03-16T10:07:42.210780+0000","last_clean":"2024-03-16T10:07:42.210780+0000","last_became_active":"2024-03-16T10:06:04.217378+0000","last_became_peered":"2024-03-16T10:06:04.217378+0000","last_unstale":"2024-03-16T10:07:42.210780+0000","last_undegraded":"2024-03-16T10:07:42.210780+0000","last_fullsized":"2024-03-16T10:07:42.210780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:35:23.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.928126+0000","last_change":"2024-03-16T10:06:08.344488+0000","last_active":"2024-03-16T10:07:13.928126+0000","last_peered":"2024-03-16T10:07:13.928126+0000","last_clean":"2024-03-16T10:07:13.928126+0000","last_became_active":"2024-03-16T10:06:08.343964+0000","last_became_peered":"2024-03-16T10:06:08.343964+0000","last_unstale":"2024-03-16T10:07:13.928126+0000","last_undegraded":"2024-03-16T10:07:13.928126+0000","last_fullsized":"2024-03-16T10:07:13.928126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:41:09.144062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":123,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:08:04.730240+0000","last_change":"2024-03-16T10:06:03.231805+0000","last_active":"2024-03-16T10:08:04.730240+0000","last_peered":"2024-03-16T10:08:04.730240+0000","last_clean":"2024-03-16T10:08:04.730240+0000","last_became_active":"2024-03-16T10:06:03.230605+0000","last_became_peered":"2024-03-16T10:06:03.230605+0000","last_unstale":"2024-03-16T10:08:04.730240+0000","last_undegraded":"2024-03-16T10:08:04.730240+0000","last_fullsized":"2024-03-16T10:08:04.730240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:21.545277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211237+0000","last_change":"2024-03-16T10:06:04.195674+0000","last_active":"2024-03-16T10:07:42.211237+0000","last_peered":"2024-03-16T10:07:42.211237+0000","last_clean":"2024-03-16T10:07:42.211237+0000","last_became_active":"2024-03-16T10:06:04.195364+0000","last_became_peered":"2024-03-16T10:06:04.195364+0000","last_unstale":"2024-03-16T10:07:42.211237+0000","last_undegraded":"2024-03-16T10:07:42.211237+0000","last_fullsized":"2024-03-16T10:07:42.211237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:40.101280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:13.933064+0000","last_change":"2024-03-16T10:06:08.342352+0000","last_active":"2024-03-16T10:07:13.933064+0000","last_peered":"2024-03-16T10:07:13.933064+0000","last_clean":"2024-03-16T10:07:13.933064+0000","last_became_active":"2024-03-16T10:06:08.341910+0000","last_became_peered":"2024-03-16T10:06:08.341910+0000","last_unstale":"2024-03-16T10:07:13.933064+0000","last_undegraded":"2024-03-16T10:07:13.933064+0000","last_fullsized":"2024-03-16T10:07:13.933064+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:48:23.446076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.196016+0000","last_change":"2024-03-16T10:06:03.236814+0000","last_active":"2024-03-16T10:07:14.196016+0000","last_peered":"2024-03-16T10:07:14.196016+0000","last_clean":"2024-03-16T10:07:14.196016+0000","last_became_active":"2024-03-16T10:06:03.236428+0000","last_became_peered":"2024-03-16T10:06:03.236428+0000","last_unstale":"2024-03-16T10:07:14.196016+0000","last_undegraded":"2024-03-16T10:07:14.196016+0000","last_fullsized":"2024-03-16T10:07:14.196016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:44.565924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.197506+0000","last_change":"2024-03-16T10:06:04.224725+0000","last_active":"2024-03-16T10:07:42.197506+0000","last_peered":"2024-03-16T10:07:42.197506+0000","last_clean":"2024-03-16T10:07:42.197506+0000","last_became_active":"2024-03-16T10:06:04.224308+0000","last_became_peered":"2024-03-16T10:06:04.224308+0000","last_unstale":"2024-03-16T10:07:42.197506+0000","last_undegraded":"2024-03-16T10:07:42.197506+0000","last_fullsized":"2024-03-16T10:07:42.197506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:24:55.128000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.195273+0000","last_change":"2024-03-16T10:06:08.330089+0000","last_active":"2024-03-16T10:07:14.195273+0000","last_peered":"2024-03-16T10:07:14.195273+0000","last_clean":"2024-03-16T10:07:14.195273+0000","last_became_active":"2024-03-16T10:06:08.329692+0000","last_became_peered":"2024-03-16T10:06:08.329692+0000","last_unstale":"2024-03-16T10:07:14.195273+0000","last_undegraded":"2024-03-16T10:07:14.195273+0000","last_fullsized":"2024-03-16T10:07:14.195273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:07.282107+0000","last_clean_scrub_stamp":"2024-03-16T10:06:07.282107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:22.141351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:14.198145+0000","last_change":"2024-03-16T10:06:03.247525+0000","last_active":"2024-03-16T10:07:14.198145+0000","last_peered":"2024-03-16T10:07:14.198145+0000","last_clean":"2024-03-16T10:07:14.198145+0000","last_became_active":"2024-03-16T10:06:03.246854+0000","last_became_peered":"2024-03-16T10:06:03.246854+0000","last_unstale":"2024-03-16T10:07:14.198145+0000","last_undegraded":"2024-03-16T10:07:14.198145+0000","last_fullsized":"2024-03-16T10:07:14.198145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:02.182077+0000","last_clean_scrub_stamp":"2024-03-16T10:06:02.182077+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:20.648750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196941+0000","last_change":"2024-03-16T10:06:04.194982+0000","last_active":"2024-03-16T10:07:42.196941+0000","last_peered":"2024-03-16T10:07:42.196941+0000","last_clean":"2024-03-16T10:07:42.196941+0000","last_became_active":"2024-03-16T10:06:04.194696+0000","last_became_peered":"2024-03-16T10:06:04.194696+0000","last_unstale":"2024-03-16T10:07:42.196941+0000","last_undegraded":"2024-03-16T10:07:42.196941+0000","last_fullsized":"2024-03-16T10:07:42.196941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:42:20.085100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.192953+0000","last_change":"2024-03-16T10:06:04.196047+0000","last_active":"2024-03-16T10:07:42.192953+0000","last_peered":"2024-03-16T10:07:42.192953+0000","last_clean":"2024-03-16T10:07:42.192953+0000","last_became_active":"2024-03-16T10:06:04.195742+0000","last_became_peered":"2024-03-16T10:06:04.195742+0000","last_unstale":"2024-03-16T10:07:42.192953+0000","last_undegraded":"2024-03-16T10:07:42.192953+0000","last_fullsized":"2024-03-16T10:07:42.192953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:00:54.239827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190649+0000","last_change":"2024-03-16T10:06:04.193481+0000","last_active":"2024-03-16T10:07:42.190649+0000","last_peered":"2024-03-16T10:07:42.190649+0000","last_clean":"2024-03-16T10:07:42.190649+0000","last_became_active":"2024-03-16T10:06:04.193145+0000","last_became_peered":"2024-03-16T10:06:04.193145+0000","last_unstale":"2024-03-16T10:07:42.190649+0000","last_undegraded":"2024-03-16T10:07:42.190649+0000","last_fullsized":"2024-03-16T10:07:42.190649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:02:10.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.209114+0000","last_change":"2024-03-16T10:06:04.223322+0000","last_active":"2024-03-16T10:07:42.209114+0000","last_peered":"2024-03-16T10:07:42.209114+0000","last_clean":"2024-03-16T10:07:42.209114+0000","last_became_active":"2024-03-16T10:06:04.223116+0000","last_became_peered":"2024-03-16T10:06:04.223116+0000","last_unstale":"2024-03-16T10:07:42.209114+0000","last_undegraded":"2024-03-16T10:07:42.209114+0000","last_fullsized":"2024-03-16T10:07:42.209114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:46.641716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.216835+0000","last_change":"2024-03-16T10:07:14.199872+0000","last_active":"2024-03-16T10:07:42.216835+0000","last_peered":"2024-03-16T10:07:42.216835+0000","last_clean":"2024-03-16T10:07:42.216835+0000","last_became_active":"2024-03-16T10:07:14.199210+0000","last_became_peered":"2024-03-16T10:07:14.199210+0000","last_unstale":"2024-03-16T10:07:42.216835+0000","last_undegraded":"2024-03-16T10:07:42.216835+0000","last_fullsized":"2024-03-16T10:07:42.216835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:42:36.253729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.214690+0000","last_change":"2024-03-16T10:06:04.225518+0000","last_active":"2024-03-16T10:07:42.214690+0000","last_peered":"2024-03-16T10:07:42.214690+0000","last_clean":"2024-03-16T10:07:42.214690+0000","last_became_active":"2024-03-16T10:06:04.225102+0000","last_became_peered":"2024-03-16T10:06:04.225102+0000","last_unstale":"2024-03-16T10:07:42.214690+0000","last_undegraded":"2024-03-16T10:07:42.214690+0000","last_fullsized":"2024-03-16T10:07:42.214690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:38:54.493238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.190560+0000","last_change":"2024-03-16T10:06:04.192978+0000","last_active":"2024-03-16T10:07:42.190560+0000","last_peered":"2024-03-16T10:07:42.190560+0000","last_clean":"2024-03-16T10:07:42.190560+0000","last_became_active":"2024-03-16T10:06:04.192641+0000","last_became_peered":"2024-03-16T10:06:04.192641+0000","last_unstale":"2024-03-16T10:07:42.190560+0000","last_undegraded":"2024-03-16T10:07:42.190560+0000","last_fullsized":"2024-03-16T10:07:42.190560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:58.334266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.202684+0000","last_change":"2024-03-16T10:06:04.190020+0000","last_active":"2024-03-16T10:07:42.202684+0000","last_peered":"2024-03-16T10:07:42.202684+0000","last_clean":"2024-03-16T10:07:42.202684+0000","last_became_active":"2024-03-16T10:06:04.189703+0000","last_became_peered":"2024-03-16T10:06:04.189703+0000","last_unstale":"2024-03-16T10:07:42.202684+0000","last_undegraded":"2024-03-16T10:07:42.202684+0000","last_fullsized":"2024-03-16T10:07:42.202684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:23.689558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.196634+0000","last_change":"2024-03-16T10:06:04.234714+0000","last_active":"2024-03-16T10:07:42.196634+0000","last_peered":"2024-03-16T10:07:42.196634+0000","last_clean":"2024-03-16T10:07:42.196634+0000","last_became_active":"2024-03-16T10:06:04.233246+0000","last_became_peered":"2024-03-16T10:06:04.233246+0000","last_unstale":"2024-03-16T10:07:42.196634+0000","last_undegraded":"2024-03-16T10:07:42.196634+0000","last_fullsized":"2024-03-16T10:07:42.196634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:13:31.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211762+0000","last_change":"2024-03-16T10:06:04.219242+0000","last_active":"2024-03-16T10:07:42.211762+0000","last_peered":"2024-03-16T10:07:42.211762+0000","last_clean":"2024-03-16T10:07:42.211762+0000","last_became_active":"2024-03-16T10:06:04.218955+0000","last_became_peered":"2024-03-16T10:06:04.218955+0000","last_unstale":"2024-03-16T10:07:42.211762+0000","last_undegraded":"2024-03-16T10:07:42.211762+0000","last_fullsized":"2024-03-16T10:07:42.211762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:29:40.760143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.203178+0000","last_change":"2024-03-16T10:06:04.231516+0000","last_active":"2024-03-16T10:07:42.203178+0000","last_peered":"2024-03-16T10:07:42.203178+0000","last_clean":"2024-03-16T10:07:42.203178+0000","last_became_active":"2024-03-16T10:06:04.230328+0000","last_became_peered":"2024-03-16T10:06:04.230328+0000","last_unstale":"2024-03-16T10:07:42.203178+0000","last_undegraded":"2024-03-16T10:07:42.203178+0000","last_fullsized":"2024-03-16T10:07:42.203178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:48:55.678921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.211721+0000","last_change":"2024-03-16T10:07:13.930195+0000","last_active":"2024-03-16T10:07:42.211721+0000","last_peered":"2024-03-16T10:07:42.211721+0000","last_clean":"2024-03-16T10:07:42.211721+0000","last_became_active":"2024-03-16T10:07:13.929801+0000","last_became_peered":"2024-03-16T10:07:13.929801+0000","last_unstale":"2024-03-16T10:07:42.211721+0000","last_undegraded":"2024-03-16T10:07:42.211721+0000","last_fullsized":"2024-03-16T10:07:42.211721+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:00:07.902956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.215327+0000","last_change":"2024-03-16T10:06:04.226025+0000","last_active":"2024-03-16T10:07:42.215327+0000","last_peered":"2024-03-16T10:07:42.215327+0000","last_clean":"2024-03-16T10:07:42.215327+0000","last_became_active":"2024-03-16T10:06:04.225657+0000","last_became_peered":"2024-03-16T10:06:04.225657+0000","last_unstale":"2024-03-16T10:07:42.215327+0000","last_undegraded":"2024-03-16T10:07:42.215327+0000","last_fullsized":"2024-03-16T10:07:42.215327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:08:15.163902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-16T10:07:42.193956+0000","last_change":"2024-03-16T10:06:04.198047+0000","last_active":"2024-03-16T10:07:42.193956+0000","last_peered":"2024-03-16T10:07:42.193956+0000","last_clean":"2024-03-16T10:07:42.193956+0000","last_became_active":"2024-03-16T10:06:04.197751+0000","last_became_peered":"2024-03-16T10:06:04.197751+0000","last_unstale":"2024-03-16T10:07:42.193956+0000","last_undegraded":"2024-03-16T10:07:42.193956+0000","last_fullsized":"2024-03-16T10:07:42.193956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:06:03.202893+0000","last_clean_scrub_stamp":"2024-03-16T10:06:03.202893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:32:52.703845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":215634,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70360,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":862,"num_read_kb":504,"num_write":250,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":211080,"data_compressed":5523,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":505049,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1154627,"data_compressed":58972,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776175,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":508431,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":504601,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1101351,"data_compressed":57131,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":499839,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":563743,"data_compressed":22487,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":501827,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":564495,"data_compressed":22487,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":499547,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":504445,"data_compressed":20646,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":1096401,"data_compressed":57131,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34847,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36485,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64626,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65378,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65142,"data_compressed":1841,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:08:07.993 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-16T10:08:07.993 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T10:08:07.993 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-16T10:08:07.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-16T10:08:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:07 smithi039 ceph-mon[31510]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:08.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:07 smithi039 ceph-mon[31510]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:08.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:07 smithi105 ceph-mon[36559]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:08.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:07 smithi105 ceph-mon[36559]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:08.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:07 smithi194 ceph-mon[36445]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-16T10:08:08.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:07 smithi194 ceph-mon[36445]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:08:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:10.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:09 smithi039 ceph-mon[31510]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:10.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:09 smithi105 ceph-mon[36559]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:10.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:09 smithi194 ceph-mon[36445]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 -- 172.21.15.39:0/3428528182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 msgr2=0x7fc91c10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 --2- 172.21.15.39:0/3428528182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c10f160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc914009fc0 tx=0x7fc91400b330 comp rx=0 tx=0).stop 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 -- 172.21.15.39:0/3428528182 shutdown_connections 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 --2- 172.21.15.39:0/3428528182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c10f160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 --2- 172.21.15.39:0/3428528182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 --2- 172.21.15.39:0/3428528182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 -- 172.21.15.39:0/3428528182 >> 172.21.15.39:0/3428528182 conn(0x7fc91c078100 msgr2=0x7fc91c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 -- 172.21.15.39:0/3428528182 shutdown_connections 2024-03-16T10:08:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.736+0000 7fc921e00700 1 -- 172.21.15.39:0/3428528182 wait complete. 2024-03-16T10:08:10.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.737+0000 7fc921e00700 1 Processor -- start 2024-03-16T10:08:10.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.737+0000 7fc921e00700 1 -- start start 2024-03-16T10:08:10.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:10.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c1125c0 con 0x7fc91c069f80 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c112440 con 0x7fc91c108320 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc921e00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c112740 con 0x7fc91c06a920 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc91aa00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc91a000700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc91aa00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38618/0 (socket says 172.21.15.39:38618) 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc91aa00700 1 -- 172.21.15.39:0/747422143 learned_addr learned my addr 172.21.15.39:0/747422143 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.738+0000 7fc91b400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 -- 172.21.15.39:0/747422143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 msgr2=0x7fc91c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 -- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 msgr2=0x7fc91c1a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 -- 172.21.15.39:0/747422143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c19b210 con 0x7fc91c06a920 2024-03-16T10:08:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91aa00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91b400700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc91a000700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c1a0e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc90800d950 tx=0x7fc90800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908016070 con 0x7fc91c06a920 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc90800fd70 con 0x7fc91c06a920 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91c19b500 con 0x7fc91c06a920 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800e3b0 con 0x7fc91c06a920 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.739+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91c1a77c0 con 0x7fc91c06a920 2024-03-16T10:08:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.741+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91c068f90 con 0x7fc91c06a920 2024-03-16T10:08:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.743+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc908019070 con 0x7fc91c06a920 2024-03-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.744+0000 7fc920a00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 0x7fc9000a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.744+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7fc9080ce790 con 0x7fc91c06a920 2024-03-16T10:08:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.744+0000 7fc91aa00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 0x7fc9000a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:10.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.745+0000 7fc91aa00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 0x7fc9000a4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc91c19bbf0 tx=0x7fc90c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:10.748+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc90809b490 con 0x7fc91c06a920 2024-03-16T10:08:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.034+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc91c19bfe0 con 0x7fc91c06a920 2024-03-16T10:08:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.035+0000 7fc920a00700 1 -- 172.21.15.39:0/747422143 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc90809ad90 con 0x7fc91c06a920 2024-03-16T10:08:11.037 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:11.037 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T10:08:11.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 msgr2=0x7fc9000a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 0x7fc9000a4dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc91c19bbf0 tx=0x7fc90c009310 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 msgr2=0x7fc91c1a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c1a0e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc90800d950 tx=0x7fc90800dc60 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 shutdown_connections 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc9000a2910 0x7fc9000a4dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc91c108320 0x7fc91c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c069f80 0x7fc91c1a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 --2- 172.21.15.39:0/747422143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc91c06a920 0x7fc91c1a0e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 >> 172.21.15.39:0/747422143 conn(0x7fc91c078100 msgr2=0x7fc91c10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 shutdown_connections 2024-03-16T10:08:11.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:11.038+0000 7fc921e00700 1 -- 172.21.15.39:0/747422143 wait complete. 2024-03-16T10:08:11.848 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-16T10:08:11.849 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 -- 172.21.15.39:0/2986643827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc81053e0 msgr2=0x7f4bc810bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 --2- 172.21.15.39:0/2986643827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc81053e0 0x7f4bc810bc70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009ca0 tx=0x7f4bc002c100 comp rx=0 tx=0).stop 2024-03-16T10:08:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 -- 172.21.15.39:0/2986643827 shutdown_connections 2024-03-16T10:08:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 --2- 172.21.15.39:0/2986643827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc81053e0 0x7f4bc810bc70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 --2- 172.21.15.39:0/2986643827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc8104a20 0x7f4bc8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 --2- 172.21.15.39:0/2986643827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8103830 0x7f4bc8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 -- 172.21.15.39:0/2986643827 >> 172.21.15.39:0/2986643827 conn(0x7f4bc80fedc0 msgr2=0x7f4bc81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:12.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.090+0000 7f4bcec00640 1 -- 172.21.15.39:0/2986643827 shutdown_connections 2024-03-16T10:08:12.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.091+0000 7f4bcec00640 1 -- 172.21.15.39:0/2986643827 wait complete. 2024-03-16T10:08:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 Processor -- start 2024-03-16T10:08:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 -- start start 2024-03-16T10:08:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8104a20 0x7f4bc8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 0x7f4bc81a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc810f4b0 con 0x7f4bc8104a20 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc810f330 con 0x7f4bc81053e0 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.092+0000 7f4bcec00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc810f630 con 0x7f4bc8103830 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8104a20 0x7f4bc8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:53830/0 (socket says 172.21.15.39:53830) 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bce200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 0x7f4bc81a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 -- 172.21.15.39:0/1821199813 learned_addr learned my addr 172.21.15.39:0/1821199813 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 -- 172.21.15.39:0/1821199813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 msgr2=0x7f4bc81a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 0x7f4bc81a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 -- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8104a20 msgr2=0x7f4bc8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8104a20 0x7f4bc8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 -- 172.21.15.39:0/1821199813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc81a1680 con 0x7f4bc8103830 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bce200640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 0x7f4bc81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.093+0000 7f4bcd800640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d950 tx=0x7f4bbc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.094+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc017020 con 0x7f4bc8103830 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.094+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc009c30 con 0x7f4bc8103830 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.094+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00f720 con 0x7f4bc8103830 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.094+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc81a1970 con 0x7f4bc8103830 2024-03-16T10:08:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.094+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc81a1e70 con 0x7f4bc8103830 2024-03-16T10:08:12.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.095+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc810aaf0 con 0x7f4bc8103830 2024-03-16T10:08:12.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.097+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4bbc01a020 con 0x7f4bc8103830 2024-03-16T10:08:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.098+0000 7f4bc6000640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 0x7f4bb00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.098+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f4bbc0cebe0 con 0x7f4bc8103830 2024-03-16T10:08:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.098+0000 7f4bcce00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 0x7f4bb00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.099+0000 7f4bcce00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 0x7f4bb00a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4bc819af70 tx=0x7f4bb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:12.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.107+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4bbc021020 con 0x7f4bc8103830 2024-03-16T10:08:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:12 smithi039 ceph-mon[31510]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:12.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:12 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/747422143' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T10:08:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.360+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bc8062480 con 0x7f4bc8103830 2024-03-16T10:08:12.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:12 smithi105 ceph-mon[36559]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:12.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:12 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/747422143' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T10:08:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.363+0000 7f4bc6000640 1 -- 172.21.15.39:0/1821199813 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24614 (secure 0 0 0) 0x7f4bbc009de0 con 0x7f4bc8103830 2024-03-16T10:08:12.365 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:12.365 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:07:13.919702+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:12.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 msgr2=0x7f4bb00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 0x7f4bb00a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4bc819af70 tx=0x7f4bb4005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 msgr2=0x7f4bc8199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00d950 tx=0x7f4bbc00de10 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 shutdown_connections 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4bb00a2880 0x7f4bb00a4d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bc81053e0 0x7f4bc81a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc8104a20 0x7f4bc8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 --2- 172.21.15.39:0/1821199813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4bc8103830 0x7f4bc8199a60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 >> 172.21.15.39:0/1821199813 conn(0x7f4bc80fedc0 msgr2=0x7f4bc8107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.366+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 shutdown_connections 2024-03-16T10:08:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.367+0000 7f4bcec00640 1 -- 172.21.15.39:0/1821199813 wait complete. 2024-03-16T10:08:12.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:08:12.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:08:12.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:12 smithi194 ceph-mon[36445]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:12.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:12 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/747422143' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T10:08:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 -- 172.21.15.39:0/451866509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 msgr2=0x7f4b941003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 --2- 172.21.15.39:0/451866509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b941003b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009900 tx=0x7f4b8402bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 -- 172.21.15.39:0/451866509 shutdown_connections 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 --2- 172.21.15.39:0/451866509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b94101290 0x7f4b94101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 --2- 172.21.15.39:0/451866509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b941008f0 0x7f4b94100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 --2- 172.21.15.39:0/451866509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b941003b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 -- 172.21.15.39:0/451866509 >> 172.21.15.39:0/451866509 conn(0x7f4b940fbb50 msgr2=0x7f4b940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 -- 172.21.15.39:0/451866509 shutdown_connections 2024-03-16T10:08:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.605+0000 7f4b93200640 1 -- 172.21.15.39:0/451866509 wait complete. 2024-03-16T10:08:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.606+0000 7f4b93200640 1 Processor -- start 2024-03-16T10:08:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 -- start start 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 0x7f4b9419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 0x7f4b941a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9410f550 con 0x7f4b94101290 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9410f3d0 con 0x7f4b941008f0 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b93200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9410f6d0 con 0x7f4b940fff90 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b91e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 0x7f4b9419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b91e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:53836/0 (socket says 172.21.15.39:53836) 2024-03-16T10:08:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.607+0000 7f4b91e00640 1 -- 172.21.15.39:0/2180071527 learned_addr learned my addr 172.21.15.39:0/2180071527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b92800640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 0x7f4b941a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 -- 172.21.15.39:0/2180071527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 msgr2=0x7f4b9419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 0x7f4b9419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 -- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 msgr2=0x7f4b941a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 0x7f4b941a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 -- 172.21.15.39:0/2180071527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b941a17e0 con 0x7f4b940fff90 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b8be00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 0x7f4b9419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b92800640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 0x7f4b941a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b91e00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b8402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.608+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84043020 con 0x7f4b940fff90 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.609+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b840054e0 con 0x7f4b940fff90 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.609+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003c40 con 0x7f4b940fff90 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.609+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941a1a70 con 0x7f4b940fff90 2024-03-16T10:08:12.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.609+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941a1fc0 con 0x7f4b940fff90 2024-03-16T10:08:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.610+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9410ae70 con 0x7f4b940fff90 2024-03-16T10:08:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.612+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4b84046020 con 0x7f4b940fff90 2024-03-16T10:08:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.613+0000 7f4b8aa00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 0x7f4b780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.613+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9071+0+0 (secure 0 0 0) 0x7f4b840f13b0 con 0x7f4b940fff90 2024-03-16T10:08:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.613+0000 7f4b8be00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 0x7f4b780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.614+0000 7f4b8be00640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 0x7f4b780a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b9419b0d0 tx=0x7f4b7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.615+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4b8404d020 con 0x7f4b940fff90 2024-03-16T10:08:12.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.860+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b9402d400 con 0x7f4b940fff90 2024-03-16T10:08:12.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.860+0000 7f4b8aa00640 1 -- 172.21.15.39:0/2180071527 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f4b840be110 con 0x7f4b940fff90 2024-03-16T10:08:12.862 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:08:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 msgr2=0x7f4b780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 0x7f4b780a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b9419b0d0 tx=0x7f4b7c005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:12.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 msgr2=0x7f4b94199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b8402c650 comp rx=0 tx=0).stop 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 shutdown_connections 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4b780a2800 0x7f4b780a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b94101290 0x7f4b941a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b941008f0 0x7f4b9419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 --2- 172.21.15.39:0/2180071527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b940fff90 0x7f4b94199bc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 >> 172.21.15.39:0/2180071527 conn(0x7f4b940fbb50 msgr2=0x7f4b940fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 shutdown_connections 2024-03-16T10:08:12.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:12.863+0000 7f4b93200640 1 -- 172.21.15.39:0/2180071527 wait complete. 2024-03-16T10:08:12.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 -- 172.21.15.39:0/221667070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c075820 msgr2=0x7f2f1c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 --2- 172.21.15.39:0/221667070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c075820 0x7f2f1c073c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009930 tx=0x7f2f0c02bc20 comp rx=0 tx=0).stop 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 -- 172.21.15.39:0/221667070 shutdown_connections 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 --2- 172.21.15.39:0/221667070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c10ddc0 0x7f2f1c1101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 --2- 172.21.15.39:0/221667070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c074240 0x7f2f1c0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 --2- 172.21.15.39:0/221667070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c075820 0x7f2f1c073c70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 -- 172.21.15.39:0/221667070 >> 172.21.15.39:0/221667070 conn(0x7f2f1c0fcd60 msgr2=0x7f2f1c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 -- 172.21.15.39:0/221667070 shutdown_connections 2024-03-16T10:08:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.122+0000 7f2f23200640 1 -- 172.21.15.39:0/221667070 wait complete. 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.123+0000 7f2f23200640 1 Processor -- start 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.123+0000 7f2f23200640 1 -- start start 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 0x7f2f1c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c10ddc0 0x7f2f1c1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c113a40 con 0x7f2f1c075820 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c1138c0 con 0x7f2f1c074240 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f23200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c113bc0 con 0x7f2f1c10ddc0 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:35960/0 (socket says 172.21.15.39:35960) 2024-03-16T10:08:13.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 -- 172.21.15.39:0/197082452 learned_addr learned my addr 172.21.15.39:0/197082452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1b400640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 0x7f2f1c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f21200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c10ddc0 0x7f2f1c1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 -- 172.21.15.39:0/197082452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c10ddc0 msgr2=0x7f2f1c1a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c10ddc0 0x7f2f1c1a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 -- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 msgr2=0x7f2f1c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 0x7f2f1c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1be00640 1 -- 172.21.15.39:0/197082452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c1a5d90 con 0x7f2f1c074240 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.124+0000 7f2f1b400640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 0x7f2f1c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f1be00640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02c7a0 tx=0x7f2f0c02c5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c041070 con 0x7f2f1c074240 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c004450 con 0x7f2f1c074240 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003e60 con 0x7f2f1c074240 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c1a6020 con 0x7f2f1c074240 2024-03-16T10:08:13.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.125+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c1a64c0 con 0x7f2f1c074240 2024-03-16T10:08:13.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.126+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1c103f90 con 0x7f2f1c074240 2024-03-16T10:08:13.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.128+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2f0c044070 con 0x7f2f1c074240 2024-03-16T10:08:13.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.129+0000 7f2f18c00640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 0x7f2f000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.129+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f2f0c0f0990 con 0x7f2f1c074240 2024-03-16T10:08:13.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.130+0000 7f2f1b400640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 0x7f2f000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.130+0000 7f2f1b400640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 0x7f2f000a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c19f680 tx=0x7f2f08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:13.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.133+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2f0c0bd620 con 0x7f2f1c074240 2024-03-16T10:08:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1821199813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2180071527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T10:08:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-16T10:08:13.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-03-16T10:08:13.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:13 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.337+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f1c0623e0 con 0x7f2f1c074240 2024-03-16T10:08:13.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.338+0000 7f2f18c00640 1 -- 172.21.15.39:0/197082452 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f2f0c0bcf20 con 0x7f2f1c074240 2024-03-16T10:08:13.339 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 msgr2=0x7f2f000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 0x7f2f000a4d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c19f680 tx=0x7f2f08005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 msgr2=0x7f2f1c19e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02c7a0 tx=0x7f2f0c02c5c0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 shutdown_connections 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f2f000a2880 0x7f2f000a4d40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f1c10ddc0 0x7f2f1c1a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f1c075820 0x7f2f1c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 --2- 172.21.15.39:0/197082452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f1c074240 0x7f2f1c19e1c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 >> 172.21.15.39:0/197082452 conn(0x7f2f1c0fcd60 msgr2=0x7f2f1c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.341+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 shutdown_connections 2024-03-16T10:08:13.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.342+0000 7f2f23200640 1 -- 172.21.15.39:0/197082452 wait complete. 2024-03-16T10:08:13.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:08:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1821199813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2180071527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T10:08:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-16T10:08:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-03-16T10:08:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:13 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1821199813' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2180071527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-03-16T10:08:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:13 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 -- 172.21.15.39:0/739803048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 msgr2=0x7fa358101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 --2- 172.21.15.39:0/739803048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358101c10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa348009900 tx=0x7fa34802bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 -- 172.21.15.39:0/739803048 shutdown_connections 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 --2- 172.21.15.39:0/739803048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581033a0 0x7fa358109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 --2- 172.21.15.39:0/739803048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581029e0 0x7fa358102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 --2- 172.21.15.39:0/739803048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358101c10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 -- 172.21.15.39:0/739803048 >> 172.21.15.39:0/739803048 conn(0x7fa3580fcd80 msgr2=0x7fa3580ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 -- 172.21.15.39:0/739803048 shutdown_connections 2024-03-16T10:08:13.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.555+0000 7fa35d600640 1 -- 172.21.15.39:0/739803048 wait complete. 2024-03-16T10:08:13.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.556+0000 7fa35d600640 1 Processor -- start 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.556+0000 7fa35d600640 1 -- start start 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.556+0000 7fa35d600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.556+0000 7fa35d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581029e0 0x7fa35819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.556+0000 7fa35d600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581033a0 0x7fa3581a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa35d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d4b0 con 0x7fa3581029e0 2024-03-16T10:08:13.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa35d600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d330 con 0x7fa3581033a0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa35d600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa35810d630 con 0x7fa3581017f0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581029e0 0x7fa35819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:53870/0 (socket says 172.21.15.39:53870) 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 -- 172.21.15.39:0/1383966738 learned_addr learned my addr 172.21.15.39:0/1383966738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa357400640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581033a0 0x7fa3581a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 -- 172.21.15.39:0/1383966738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581033a0 msgr2=0x7fa3581a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581033a0 0x7fa3581a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 -- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581029e0 msgr2=0x7fa35819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581029e0 0x7fa35819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.557+0000 7fa356a00640 1 -- 172.21.15.39:0/1383966738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3581a19c0 con 0x7fa3581017f0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa356a00640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa34802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348043020 con 0x7fa3581017f0 2024-03-16T10:08:13.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3480054e0 con 0x7fa3581017f0 2024-03-16T10:08:13.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3581a1c50 con 0x7fa3581017f0 2024-03-16T10:08:13.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348003c40 con 0x7fa3581017f0 2024-03-16T10:08:13.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.558+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a21a0 con 0x7fa3581017f0 2024-03-16T10:08:13.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.560+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa358068a20 con 0x7fa3581017f0 2024-03-16T10:08:13.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.561+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa348046020 con 0x7fa3581017f0 2024-03-16T10:08:13.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.562+0000 7fa34f400640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 0x7fa33c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:13.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.563+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fa3480f0e20 con 0x7fa3581017f0 2024-03-16T10:08:13.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.563+0000 7fa356000640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 0x7fa33c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:13.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.564+0000 7fa356000640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 0x7fa33c0a4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa344009bd0 tx=0x7fa344005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:13.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.566+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa34804d020 con 0x7fa3581017f0 2024-03-16T10:08:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.774+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3581a2b10 con 0x7fa3581017f0 2024-03-16T10:08:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.775+0000 7fa34f400640 1 -- 172.21.15.39:0/1383966738 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa3480bdaf0 con 0x7fa3581017f0 2024-03-16T10:08:13.776 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:08:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.778+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 msgr2=0x7fa33c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.778+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 0x7fa33c0a4d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa344009bd0 tx=0x7fa344005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.778+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 msgr2=0x7fa358199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.778+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa34802c650 comp rx=0 tx=0).stop 2024-03-16T10:08:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 shutdown_connections 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fa33c0a2880 0x7fa33c0a4d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3581033a0 0x7fa3581a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3581029e0 0x7fa35819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 --2- 172.21.15.39:0/1383966738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3581017f0 0x7fa358199db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 >> 172.21.15.39:0/1383966738 conn(0x7fa3580fcd80 msgr2=0x7fa358073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 shutdown_connections 2024-03-16T10:08:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:13.779+0000 7fa35d600640 1 -- 172.21.15.39:0/1383966738 wait complete. 2024-03-16T10:08:13.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 -- 172.21.15.39:0/1856592045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 msgr2=0x7f905810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 --2- 172.21.15.39:0/1856592045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f905810bb30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9050009ca0 tx=0x7f905002c190 comp rx=0 tx=0).stop 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 -- 172.21.15.39:0/1856592045 shutdown_connections 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 --2- 172.21.15.39:0/1856592045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f905810bb30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 --2- 172.21.15.39:0/1856592045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 0x7f9058074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 --2- 172.21.15.39:0/1856592045 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f9058073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 -- 172.21.15.39:0/1856592045 >> 172.21.15.39:0/1856592045 conn(0x7f90580fcd80 msgr2=0x7f90580ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 -- 172.21.15.39:0/1856592045 shutdown_connections 2024-03-16T10:08:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.039+0000 7f905d200640 1 -- 172.21.15.39:0/1856592045 wait complete. 2024-03-16T10:08:14.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 Processor -- start 2024-03-16T10:08:14.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 -- start start 2024-03-16T10:08:14.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 0x7f9058199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f90581a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.040+0000 7f905d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f905810f680 con 0x7f90580741c0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f905d200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f905810f500 con 0x7f90581096a0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f905d200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f905810f800 con 0x7f90580757a0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9056800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f90581a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:53892/0 (socket says 172.21.15.39:53892) 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 -- 172.21.15.39:0/123960005 learned_addr learned my addr 172.21.15.39:0/123960005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 0x7f9058199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 -- 172.21.15.39:0/123960005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 msgr2=0x7f90581a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f90581a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 -- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 msgr2=0x7f9058199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 0x7f9058199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9055400640 1 -- 172.21.15.39:0/123960005 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581a1920 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.041+0000 7f9056800640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f90581a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f9055400640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9f0 tx=0x7f904400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904400cd80 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044010910 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044018990 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581a1c10 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.042+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581a2110 con 0x7f90580757a0 2024-03-16T10:08:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.043+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9058108460 con 0x7f90580757a0 2024-03-16T10:08:14.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.047+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9044010ac0 con 0x7f90580757a0 2024-03-16T10:08:14.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.049+0000 7f904ea00640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 0x7f903c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.049+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f90440cf170 con 0x7f90580757a0 2024-03-16T10:08:14.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.049+0000 7f9055e00640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 0x7f903c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.050+0000 7f9055e00640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 0x7f903c0a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f905819b1e0 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:14.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.052+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f904409b7f0 con 0x7f90580757a0 2024-03-16T10:08:14.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.327+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9058062480 con 0x7f90580757a0 2024-03-16T10:08:14.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.327+0000 7f904ea00640 1 -- 172.21.15.39:0/123960005 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f904409f890 con 0x7f90580757a0 2024-03-16T10:08:14.329 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 msgr2=0x7f903c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 0x7f903c0a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f905819b1e0 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 msgr2=0x7f905819a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9f0 tx=0x7f904400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 shutdown_connections 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f903c0a2880 0x7f903c0a4d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581096a0 0x7f90581a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.329+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90580757a0 0x7f905819a250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.330+0000 7f905d200640 1 --2- 172.21.15.39:0/123960005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90580741c0 0x7f9058199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.330+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 >> 172.21.15.39:0/123960005 conn(0x7f90580fcd80 msgr2=0x7f9058105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.330+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 shutdown_connections 2024-03-16T10:08:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.330+0000 7f905d200640 1 -- 172.21.15.39:0/123960005 wait complete. 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T10:08:14.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/197082452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1383966738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:14 smithi039 ceph-mon[31510]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T10:08:14.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:08:14.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/197082452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1383966738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:14 smithi105 ceph-mon[36559]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.11", "id": [5, 8]}]': finished 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: osdmap e93: 12 total, 12 up, 12 in 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/197082452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1383966738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:14.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:14 smithi194 ceph-mon[36445]: osdmap e94: 12 total, 12 up, 12 in 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 -- 172.21.15.39:0/3418958520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b54101600 msgr2=0x7f8b54101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 --2- 172.21.15.39:0/3418958520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b54101600 0x7f8b54101a20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009900 tx=0x7f8b4402bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 -- 172.21.15.39:0/3418958520 shutdown_connections 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 --2- 172.21.15.39:0/3418958520 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 0x7f8b54109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 --2- 172.21.15.39:0/3418958520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b541027f0 0x7f8b54102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 --2- 172.21.15.39:0/3418958520 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b54101600 0x7f8b54101a20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 -- 172.21.15.39:0/3418958520 >> 172.21.15.39:0/3418958520 conn(0x7f8b540fcb70 msgr2=0x7f8b540fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 -- 172.21.15.39:0/3418958520 shutdown_connections 2024-03-16T10:08:14.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.538+0000 7f8b59000640 1 -- 172.21.15.39:0/3418958520 wait complete. 2024-03-16T10:08:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.539+0000 7f8b59000640 1 Processor -- start 2024-03-16T10:08:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.539+0000 7f8b59000640 1 -- start start 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 0x7f8b541978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 0x7f8b5419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5410d2c0 con 0x7f8b54101600 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5410d140 con 0x7f8b541027f0 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b59000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5410d440 con 0x7f8b541031b0 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 0x7f8b541978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36026/0 (socket says 172.21.15.39:36026) 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b4be00640 1 -- 172.21.15.39:0/2998165834 learned_addr learned my addr 172.21.15.39:0/2998165834 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53e00640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 0x7f8b5419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 -- 172.21.15.39:0/2998165834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 msgr2=0x7f8b5419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 0x7f8b5419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 -- 172.21.15.39:0/2998165834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 msgr2=0x7f8b54197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b53400640 1 -- 172.21.15.39:0/2998165834 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5419f4f0 con 0x7f8b54101600 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.540+0000 7f8b4be00640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b53400640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 0x7f8b541978d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009230 tx=0x7f8b4402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44043020 con 0x7f8b54101600 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b440054e0 con 0x7f8b54101600 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44003c60 con 0x7f8b54101600 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5419f780 con 0x7f8b54101600 2024-03-16T10:08:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.541+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5419fca0 con 0x7f8b54101600 2024-03-16T10:08:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.543+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b54108920 con 0x7f8b54101600 2024-03-16T10:08:14.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.544+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8b44046020 con 0x7f8b54101600 2024-03-16T10:08:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.547+0000 7f8b50c00640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 0x7f8b380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.547+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f8b440f0ec0 con 0x7f8b54101600 2024-03-16T10:08:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.547+0000 7f8b4be00640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 0x7f8b380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.548+0000 7f8b4be00640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 0x7f8b380a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b54198de0 tx=0x7f8b3c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:14.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.552+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8b4404d020 con 0x7f8b54101600 2024-03-16T10:08:14.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.850+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b5402d1b0 con 0x7f8b54101600 2024-03-16T10:08:14.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.851+0000 7f8b50c00640 1 -- 172.21.15.39:0/2998165834 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f8b440bdb50 con 0x7f8b54101600 2024-03-16T10:08:14.852 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:14.852 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:08:14.853 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 msgr2=0x7f8b380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 0x7f8b380a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b54198de0 tx=0x7f8b3c00b4a0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 msgr2=0x7f8b541978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 0x7f8b541978d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009230 tx=0x7f8b4402c6c0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 shutdown_connections 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f8b380a2800 0x7f8b380a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b541031b0 0x7f8b5419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b541027f0 0x7f8b54197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 --2- 172.21.15.39:0/2998165834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54101600 0x7f8b541978d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 >> 172.21.15.39:0/2998165834 conn(0x7f8b540fcb70 msgr2=0x7f8b54105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:14.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.853+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 shutdown_connections 2024-03-16T10:08:14.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:14.854+0000 7f8b59000640 1 -- 172.21.15.39:0/2998165834 wait complete. 2024-03-16T10:08:14.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:15.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 -- 172.21.15.39:0/3238384704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41031a0 msgr2=0x7f9ce4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 --2- 172.21.15.39:0/3238384704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41031a0 0x7f9ce4109a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009ca0 tx=0x7f9cdc02c100 comp rx=0 tx=0).stop 2024-03-16T10:08:15.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 -- 172.21.15.39:0/3238384704 shutdown_connections 2024-03-16T10:08:15.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 --2- 172.21.15.39:0/3238384704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41031a0 0x7f9ce4109a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 --2- 172.21.15.39:0/3238384704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41027e0 0x7f9ce4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 --2- 172.21.15.39:0/3238384704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41015f0 0x7f9ce4101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.086+0000 7f9cea800640 1 -- 172.21.15.39:0/3238384704 >> 172.21.15.39:0/3238384704 conn(0x7f9ce40fcb40 msgr2=0x7f9ce40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:15.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.087+0000 7f9cea800640 1 -- 172.21.15.39:0/3238384704 shutdown_connections 2024-03-16T10:08:15.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.087+0000 7f9cea800640 1 -- 172.21.15.39:0/3238384704 wait complete. 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.087+0000 7f9cea800640 1 Processor -- start 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 -- start start 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41027e0 0x7f9ce4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 0x7f9ce4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410d270 con 0x7f9ce41031a0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410d0f0 con 0x7f9ce41015f0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9cea800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410d3f0 con 0x7f9ce41027e0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9ce9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41027e0 0x7f9ce4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9ce9400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36038/0 (socket says 172.21.15.39:36038) 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9ce9400640 1 -- 172.21.15.39:0/2938079867 learned_addr learned my addr 172.21.15.39:0/2938079867 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.088+0000 7f9ce9e00640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 0x7f9ce4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 -- 172.21.15.39:0/2938079867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41027e0 msgr2=0x7f9ce4073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41027e0 0x7f9ce4073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 -- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 msgr2=0x7f9ce4074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 0x7f9ce4074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 -- 172.21.15.39:0/2938079867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce40749a0 con 0x7f9ce41015f0 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9e00640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 0x7f9ce4074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:15.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce9400640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e9c0 tx=0x7f9cd800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800cd80 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cd80035f0 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41a1940 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8010690 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.089+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a1ee0 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.091+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce41088b0 con 0x7f9ce41015f0 2024-03-16T10:08:15.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.092+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9cd80037a0 con 0x7f9ce41015f0 2024-03-16T10:08:15.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.095+0000 7f9ce2000640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 0x7f9ccc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.095+0000 7f9ce8a00640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 0x7f9ccc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.095+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f9cd80cf2b0 con 0x7f9ce41015f0 2024-03-16T10:08:15.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.096+0000 7f9ce8a00640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 0x7f9ccc0a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4075300 tx=0x7f9cd000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:15.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.097+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9cd809bf40 con 0x7f9ce41015f0 2024-03-16T10:08:15.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:15 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/123960005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:15.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:15 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2998165834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:15.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.295+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ce4062480 con 0x7f9ce41015f0 2024-03-16T10:08:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.297+0000 7f9ce2000640 1 -- 172.21.15.39:0/2938079867 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24850 (secure 0 0 0) 0x7f9cd809b840 con 0x7f9ce41015f0 2024-03-16T10:08:15.299 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:15.299 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:08:14.004515+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 msgr2=0x7f9ccc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 0x7f9ccc0a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4075300 tx=0x7f9cd000b390 comp rx=0 tx=0).stop 2024-03-16T10:08:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 msgr2=0x7f9ce4075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800e9c0 tx=0x7f9cd800ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 shutdown_connections 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9ccc0a2880 0x7f9ccc0a4d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ce41031a0 0x7f9ce4074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ce41027e0 0x7f9ce4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 --2- 172.21.15.39:0/2938079867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ce41015f0 0x7f9ce4075590 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 >> 172.21.15.39:0/2938079867 conn(0x7f9ce40fcb40 msgr2=0x7f9ce4105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 shutdown_connections 2024-03-16T10:08:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.300+0000 7f9cea800640 1 -- 172.21.15.39:0/2938079867 wait complete. 2024-03-16T10:08:15.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:08:15.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:08:15.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:15 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/123960005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:15.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:15 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2998165834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:15.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:15 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/123960005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:15.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:15 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2998165834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.528+0000 7f4512600640 1 -- 172.21.15.39:0/3799656954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c1005c0 msgr2=0x7f450c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.528+0000 7f4512600640 1 --2- 172.21.15.39:0/3799656954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c1005c0 0x7f450c1009e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4500009900 tx=0x7f450002bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 -- 172.21.15.39:0/3799656954 shutdown_connections 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 --2- 172.21.15.39:0/3799656954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c102170 0x7f450c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 --2- 172.21.15.39:0/3799656954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1017b0 0x7f450c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 --2- 172.21.15.39:0/3799656954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c1005c0 0x7f450c1009e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 -- 172.21.15.39:0/3799656954 >> 172.21.15.39:0/3799656954 conn(0x7f450c0fbb50 msgr2=0x7f450c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 -- 172.21.15.39:0/3799656954 shutdown_connections 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.529+0000 7f4512600640 1 -- 172.21.15.39:0/3799656954 wait complete. 2024-03-16T10:08:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.530+0000 7f4512600640 1 Processor -- start 2024-03-16T10:08:15.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 -- start start 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c1017b0 0x7f450c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 0x7f450c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f450c10c290 con 0x7f450c1017b0 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f450c10c110 con 0x7f450c1005c0 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.531+0000 7f4512600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f450c10c410 con 0x7f450c102170 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36066/0 (socket says 172.21.15.39:36066) 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 -- 172.21.15.39:0/3631293024 learned_addr learned my addr 172.21.15.39:0/3631293024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511c00640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 0x7f450c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f450be00640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c1017b0 0x7f450c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 -- 172.21.15.39:0/3631293024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 msgr2=0x7f450c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 0x7f450c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 -- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c1017b0 msgr2=0x7f450c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c1017b0 0x7f450c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 -- 172.21.15.39:0/3631293024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c19f660 con 0x7f450c1005c0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511c00640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 0x7f450c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4511200640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f450002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.532+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500043020 con 0x7f450c1005c0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.533+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45000054e0 con 0x7f450c1005c0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.533+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c19f890 con 0x7f450c1005c0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.533+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500003c40 con 0x7f450c1005c0 2024-03-16T10:08:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.533+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c19fde0 con 0x7f450c1005c0 2024-03-16T10:08:15.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.534+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c1078e0 con 0x7f450c1005c0 2024-03-16T10:08:15.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.536+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4500046020 con 0x7f450c1005c0 2024-03-16T10:08:15.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.537+0000 7f4509600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 0x7f44f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:15.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.537+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f45000f0fb0 con 0x7f450c1005c0 2024-03-16T10:08:15.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.537+0000 7f450be00640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 0x7f44f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:15.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.538+0000 7f450be00640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 0x7f44f40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f450c069220 tx=0x7f44f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:15.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.544+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f450004d020 con 0x7f450c1005c0 2024-03-16T10:08:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.747+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f450c02d3d0 con 0x7f450c1005c0 2024-03-16T10:08:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.747+0000 7f4509600640 1 -- 172.21.15.39:0/3631293024 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f450003c350 con 0x7f450c1005c0 2024-03-16T10:08:15.749 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:08:15.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.750+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 msgr2=0x7f44f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.750+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 0x7f44f40a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f450c069220 tx=0x7f44f8005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:15.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 msgr2=0x7f450c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:15.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f450002c650 comp rx=0 tx=0).stop 2024-03-16T10:08:15.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 shutdown_connections 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f44f40a2800 0x7f44f40a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f450c102170 0x7f450c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f450c1017b0 0x7f450c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 --2- 172.21.15.39:0/3631293024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1005c0 0x7f450c0694b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 >> 172.21.15.39:0/3631293024 conn(0x7f450c0fbb50 msgr2=0x7f450c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 shutdown_connections 2024-03-16T10:08:15.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:15.751+0000 7f4512600640 1 -- 172.21.15.39:0/3631293024 wait complete. 2024-03-16T10:08:15.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:08:16.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 -- 172.21.15.39:0/4249229050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c01021f0 msgr2=0x7f05c01045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 --2- 172.21.15.39:0/4249229050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c01021f0 0x7f05c01045e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009900 tx=0x7f05b002bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 -- 172.21.15.39:0/4249229050 shutdown_connections 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 --2- 172.21.15.39:0/4249229050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c0107450 0x7f05c0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 --2- 172.21.15.39:0/4249229050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c0106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 --2- 172.21.15.39:0/4249229050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c01021f0 0x7f05c01045e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 -- 172.21.15.39:0/4249229050 >> 172.21.15.39:0/4249229050 conn(0x7f05c00fbb20 msgr2=0x7f05c00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 -- 172.21.15.39:0/4249229050 shutdown_connections 2024-03-16T10:08:16.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.009+0000 7f05c5a00640 1 -- 172.21.15.39:0/4249229050 wait complete. 2024-03-16T10:08:16.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.010+0000 7f05c5a00640 1 Processor -- start 2024-03-16T10:08:16.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 -- start start 2024-03-16T10:08:16.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c01021f0 0x7f05c0199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 0x7f05c01a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010d390 con 0x7f05c01021f0 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010d210 con 0x7f05c0104b20 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010d510 con 0x7f05c0107450 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c01021f0 0x7f05c0199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05bf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05bf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36078/0 (socket says 172.21.15.39:36078) 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05bf400640 1 -- 172.21.15.39:0/678733253 learned_addr learned my addr 172.21.15.39:0/678733253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.011+0000 7f05c5000640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 0x7f05c01a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 -- 172.21.15.39:0/678733253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 msgr2=0x7f05c01a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 0x7f05c01a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 -- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c01021f0 msgr2=0x7f05c0199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c01021f0 0x7f05c0199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 -- 172.21.15.39:0/678733253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a1830 con 0x7f05c0104b20 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05c5000640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 0x7f05c01a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bf400640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00e9a0 tx=0x7f05ac00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00cd80 con 0x7f05c0104b20 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05ac0035f0 con 0x7f05c0104b20 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac010670 con 0x7f05c0104b20 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.012+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a1b20 con 0x7f05c0104b20 2024-03-16T10:08:16.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.013+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a2040 con 0x7f05c0104b20 2024-03-16T10:08:16.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.014+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c0108f40 con 0x7f05c0104b20 2024-03-16T10:08:16.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.017+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f05ac0037a0 con 0x7f05c0104b20 2024-03-16T10:08:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.018+0000 7f05bcc00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 0x7f05a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.018+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f05ac0cf170 con 0x7f05c0104b20 2024-03-16T10:08:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.018+0000 7f05bfe00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 0x7f05a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.019+0000 7f05bfe00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 0x7f05a40a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05b002c250 tx=0x7f05b002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.026+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05ac09be00 con 0x7f05c0104b20 2024-03-16T10:08:16.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.235+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05c00623e0 con 0x7f05c0104b20 2024-03-16T10:08:16.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.235+0000 7f05bcc00640 1 -- 172.21.15.39:0/678733253 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f05ac09b700 con 0x7f05c0104b20 2024-03-16T10:08:16.237 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 msgr2=0x7f05a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 0x7f05a40a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f05b002c250 tx=0x7f05b002cf90 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 msgr2=0x7f05c019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00e9a0 tx=0x7f05ac00ee60 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 shutdown_connections 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f05a40a2800 0x7f05a40a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05c0107450 0x7f05c01a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0104b20 0x7f05c019a160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 --2- 172.21.15.39:0/678733253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05c01021f0 0x7f05c0199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 >> 172.21.15.39:0/678733253 conn(0x7f05c00fbb20 msgr2=0x7f05c01019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 shutdown_connections 2024-03-16T10:08:16.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.238+0000 7f05c5a00640 1 -- 172.21.15.39:0/678733253 wait complete. 2024-03-16T10:08:16.260 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:08:16.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:16 smithi039 ceph-mon[31510]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:16.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:16 smithi039 ceph-mon[31510]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T10:08:16.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:16 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2938079867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:16.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:16 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3631293024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:16 smithi105 ceph-mon[36559]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:16 smithi105 ceph-mon[36559]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T10:08:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:16 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2938079867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:16 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3631293024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:16.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:16 smithi194 ceph-mon[36445]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:16.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:16 smithi194 ceph-mon[36445]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T10:08:16.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:16 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2938079867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:16.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:16 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3631293024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 -- 172.21.15.39:0/4236674052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe841055d0 msgr2=0x7efe8410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 --2- 172.21.15.39:0/4236674052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe841055d0 0x7efe8410be60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009ca0 tx=0x7efe7c02c100 comp rx=0 tx=0).stop 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 -- 172.21.15.39:0/4236674052 shutdown_connections 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 --2- 172.21.15.39:0/4236674052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe841055d0 0x7efe8410be60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 --2- 172.21.15.39:0/4236674052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84104c10 0x7efe84105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 --2- 172.21.15.39:0/4236674052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe84103a20 0x7efe84103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 -- 172.21.15.39:0/4236674052 >> 172.21.15.39:0/4236674052 conn(0x7efe840fef90 msgr2=0x7efe841013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 -- 172.21.15.39:0/4236674052 shutdown_connections 2024-03-16T10:08:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.477+0000 7efe8c600640 1 -- 172.21.15.39:0/4236674052 wait complete. 2024-03-16T10:08:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.478+0000 7efe8c600640 1 Processor -- start 2024-03-16T10:08:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.478+0000 7efe8c600640 1 -- start start 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe84104c10 0x7efe8419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 0x7efe841a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410f680 con 0x7efe84104c10 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410f500 con 0x7efe84103a20 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8c600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410f800 con 0x7efe841055d0 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36094/0 (socket says 172.21.15.39:36094) 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 -- 172.21.15.39:0/3260572639 learned_addr learned my addr 172.21.15.39:0/3260572639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89200640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe84104c10 0x7efe8419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8a600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 0x7efe841a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 -- 172.21.15.39:0/3260572639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 msgr2=0x7efe841a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 0x7efe841a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 -- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe84104c10 msgr2=0x7efe8419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe84104c10 0x7efe8419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe89c00640 1 -- 172.21.15.39:0/3260572639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe841a1900 con 0x7efe84103a20 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.479+0000 7efe8a600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 0x7efe841a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe89c00640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e9c0 tx=0x7efe7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7800cd80 con 0x7efe84103a20 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe841a1bf0 con 0x7efe84103a20 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe780035f0 con 0x7efe84103a20 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78010690 con 0x7efe84103a20 2024-03-16T10:08:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.480+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe841a2140 con 0x7efe84103a20 2024-03-16T10:08:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.481+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8410ad60 con 0x7efe84103a20 2024-03-16T10:08:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.483+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efe780037a0 con 0x7efe84103a20 2024-03-16T10:08:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.484+0000 7efe82000640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 0x7efe6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.485+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7efe780cf2d0 con 0x7efe84103a20 2024-03-16T10:08:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.485+0000 7efe89200640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 0x7efe6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.485+0000 7efe89200640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 0x7efe6c0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe740095f0 tx=0x7efe74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.493+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe7809bf60 con 0x7efe84103a20 2024-03-16T10:08:16.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.697+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe84062480 con 0x7efe84103a20 2024-03-16T10:08:16.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.698+0000 7efe82000640 1 -- 172.21.15.39:0/3260572639 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7efe7809b860 con 0x7efe84103a20 2024-03-16T10:08:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 msgr2=0x7efe6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 0x7efe6c0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe740095f0 tx=0x7efe74005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 msgr2=0x7efe84199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e9c0 tx=0x7efe7800ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 shutdown_connections 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efe6c0a2800 0x7efe6c0a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe841055d0 0x7efe841a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe84104c10 0x7efe8419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 --2- 172.21.15.39:0/3260572639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe84103a20 0x7efe84199cc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.700+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 >> 172.21.15.39:0/3260572639 conn(0x7efe840fef90 msgr2=0x7efe84107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:16.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.701+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 shutdown_connections 2024-03-16T10:08:16.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.701+0000 7efe8c600640 1 -- 172.21.15.39:0/3260572639 wait complete. 2024-03-16T10:08:16.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.938+0000 7f6987200640 1 -- 172.21.15.39:0/186888105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988104ad0 msgr2=0x7f6988104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.938+0000 7f6987200640 1 --2- 172.21.15.39:0/186888105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988104ad0 0x7f6988104f50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6974009900 tx=0x7f697402bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 -- 172.21.15.39:0/186888105 shutdown_connections 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 --2- 172.21.15.39:0/186888105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988105490 0x7f698810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 --2- 172.21.15.39:0/186888105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988104ad0 0x7f6988104f50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 --2- 172.21.15.39:0/186888105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 -- 172.21.15.39:0/186888105 >> 172.21.15.39:0/186888105 conn(0x7f69880fee50 msgr2=0x7f6988101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 -- 172.21.15.39:0/186888105 shutdown_connections 2024-03-16T10:08:16.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.939+0000 7f6987200640 1 -- 172.21.15.39:0/186888105 wait complete. 2024-03-16T10:08:16.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 Processor -- start 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 -- start start 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988104ad0 0x7f698819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988105490 0x7f69881a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f590 con 0x7f6988104ad0 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f410 con 0x7f69881038e0 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6987200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f710 con 0x7f6988105490 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6985e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6986800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988105490 0x7f69881a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6985e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36108/0 (socket says 172.21.15.39:36108) 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6985e00640 1 -- 172.21.15.39:0/2608053805 learned_addr learned my addr 172.21.15.39:0/2608053805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:16.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.940+0000 7f6985400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988104ad0 0x7f698819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 -- 172.21.15.39:0/2608053805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988105490 msgr2=0x7f69881a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988105490 0x7f69881a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 -- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988104ad0 msgr2=0x7f698819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988104ad0 0x7f698819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 -- 172.21.15.39:0/2608053805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a1830 con 0x7f69881038e0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6985e00640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9c0 tx=0x7f697800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800cd80 con 0x7f69881038e0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69780035f0 con 0x7f69881038e0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a1b20 con 0x7f69881038e0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.941+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978010690 con 0x7f69881038e0 2024-03-16T10:08:16.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.942+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a20c0 con 0x7f69881038e0 2024-03-16T10:08:16.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.943+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698810ac00 con 0x7f69881038e0 2024-03-16T10:08:16.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.945+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f69780037a0 con 0x7f69881038e0 2024-03-16T10:08:16.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.946+0000 7f697ea00640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 0x7f696c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:16.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.946+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f69780cf210 con 0x7f69881038e0 2024-03-16T10:08:16.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.946+0000 7f6985400640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 0x7f696c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:16.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.947+0000 7f6985400640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 0x7f696c0a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f697402c240 tx=0x7f697402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:16.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:16.950+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f697809bea0 con 0x7f69881038e0 2024-03-16T10:08:17.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.159+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f69880623e0 con 0x7f69881038e0 2024-03-16T10:08:17.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.160+0000 7f697ea00640 1 -- 172.21.15.39:0/2608053805 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f697809b7a0 con 0x7f69881038e0 2024-03-16T10:08:17.162 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:17.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 msgr2=0x7f696c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 0x7f696c0a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f697402c240 tx=0x7f697402df90 comp rx=0 tx=0).stop 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 msgr2=0x7f6988199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f697800e9c0 tx=0x7f697800ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 shutdown_connections 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f696c0a2880 0x7f696c0a4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6988105490 0x7f69881a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6988104ad0 0x7f698819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 --2- 172.21.15.39:0/2608053805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69881038e0 0x7f6988199c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 >> 172.21.15.39:0/2608053805 conn(0x7f69880fee50 msgr2=0x7f6988107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 shutdown_connections 2024-03-16T10:08:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.163+0000 7f6987200640 1 -- 172.21.15.39:0/2608053805 wait complete. 2024-03-16T10:08:17.183 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:08:17.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:17 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/678733253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:17 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3260572639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 -- 172.21.15.39:0/2048128208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 msgr2=0x7f0eb0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 --2- 172.21.15.39:0/2048128208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0101a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009900 tx=0x7f0ea402bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 -- 172.21.15.39:0/2048128208 shutdown_connections 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 --2- 172.21.15.39:0/2048128208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 0x7f0eb0109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 --2- 172.21.15.39:0/2048128208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 0x7f0eb0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 --2- 172.21.15.39:0/2048128208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0101a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 -- 172.21.15.39:0/2048128208 >> 172.21.15.39:0/2048128208 conn(0x7f0eb00fcb70 msgr2=0x7f0eb00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:17.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 -- 172.21.15.39:0/2048128208 shutdown_connections 2024-03-16T10:08:17.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.390+0000 7f0eb6000640 1 -- 172.21.15.39:0/2048128208 wait complete. 2024-03-16T10:08:17.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 Processor -- start 2024-03-16T10:08:17.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 -- start start 2024-03-16T10:08:17.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 0x7f0eb019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 0x7f0eb01a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010d2b0 con 0x7f0eb01027d0 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010d130 con 0x7f0eb01015e0 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb6000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010d430 con 0x7f0eb0103190 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb4c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb4c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36128/0 (socket says 172.21.15.39:36128) 2024-03-16T10:08:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb4c00640 1 -- 172.21.15.39:0/506290882 learned_addr learned my addr 172.21.15.39:0/506290882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eafe00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 0x7f0eb019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.392+0000 7f0eb5600640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 0x7f0eb01a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 -- 172.21.15.39:0/506290882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 msgr2=0x7f0eb01a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 0x7f0eb01a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 -- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 msgr2=0x7f0eb019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 0x7f0eb019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 -- 172.21.15.39:0/506290882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb01a17e0 con 0x7f0eb01015e0 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb4c00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ea402c8f0 tx=0x7f0ea402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4041070 con 0x7f0eb01015e0 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4004450 con 0x7f0eb01015e0 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb01a1a70 con 0x7f0eb01015e0 2024-03-16T10:08:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.393+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4034440 con 0x7f0eb01015e0 2024-03-16T10:08:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.394+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb01a1fb0 con 0x7f0eb01015e0 2024-03-16T10:08:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.395+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb01a21e0 con 0x7f0eb01015e0 2024-03-16T10:08:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.398+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f0ea4044070 con 0x7f0eb01015e0 2024-03-16T10:08:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.400+0000 7f0ead600640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 0x7f0e980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.400+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f0ea40f0930 con 0x7f0eb01015e0 2024-03-16T10:08:17.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.400+0000 7f0eafe00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 0x7f0e980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.401+0000 7f0eafe00640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 0x7f0e980a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0eb019b0d0 tx=0x7f0e9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:17.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.403+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ea40bcf70 con 0x7f0eb01015e0 2024-03-16T10:08:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:17 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/678733253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:17 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3260572639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:17 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/678733253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:17 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3260572639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.705+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0eb01a23b0 con 0x7f0eb01015e0 2024-03-16T10:08:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.706+0000 7f0ead600640 1 -- 172.21.15.39:0/506290882 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f0ea40c1010 con 0x7f0eb01015e0 2024-03-16T10:08:17.708 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:17.708 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:08:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 msgr2=0x7f0e980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 0x7f0e980a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0eb019b0d0 tx=0x7f0e9c005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 msgr2=0x7f0eb0199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ea402c8f0 tx=0x7f0ea402c660 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 shutdown_connections 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f0e980a2880 0x7f0e980a4d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0eb0103190 0x7f0eb01a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0eb01027d0 0x7f0eb019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 --2- 172.21.15.39:0/506290882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb01015e0 0x7f0eb0199bc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 >> 172.21.15.39:0/506290882 conn(0x7f0eb00fcb70 msgr2=0x7f0eb0073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.709+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 shutdown_connections 2024-03-16T10:08:17.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.710+0000 7f0eb6000640 1 -- 172.21.15.39:0/506290882 wait complete. 2024-03-16T10:08:17.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.973+0000 7f9f23e00640 1 -- 172.21.15.39:0/4272939804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24102070 msgr2=0x7f9f24104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.973+0000 7f9f23e00640 1 --2- 172.21.15.39:0/4272939804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24102070 0x7f9f24104460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009900 tx=0x7f9f1402bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.973+0000 7f9f23e00640 1 -- 172.21.15.39:0/4272939804 shutdown_connections 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.973+0000 7f9f23e00640 1 --2- 172.21.15.39:0/4272939804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f241072d0 0x7f9f24109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.973+0000 7f9f23e00640 1 --2- 172.21.15.39:0/4272939804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.974+0000 7f9f23e00640 1 --2- 172.21.15.39:0/4272939804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24102070 0x7f9f24104460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.974+0000 7f9f23e00640 1 -- 172.21.15.39:0/4272939804 >> 172.21.15.39:0/4272939804 conn(0x7f9f240fba60 msgr2=0x7f9f240fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:17.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.974+0000 7f9f23e00640 1 -- 172.21.15.39:0/4272939804 shutdown_connections 2024-03-16T10:08:17.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.974+0000 7f9f23e00640 1 -- 172.21.15.39:0/4272939804 wait complete. 2024-03-16T10:08:17.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.976+0000 7f9f23e00640 1 Processor -- start 2024-03-16T10:08:17.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.976+0000 7f9f23e00640 1 -- start start 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.976+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 0x7f9f24101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.976+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.976+0000 7f9f23e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f241072d0 0x7f9f24100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f23e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d260 con 0x7f9f24102070 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f23e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d0e0 con 0x7f9f241049a0 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f23e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410d3e0 con 0x7f9f241072d0 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 0x7f9f24101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 0x7f9f24101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38858/0 (socket says 172.21.15.39:38858) 2024-03-16T10:08:17.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36148/0 (socket says 172.21.15.39:36148) 2024-03-16T10:08:17.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f22a00640 1 -- 172.21.15.39:0/2216557191 learned_addr learned my addr 172.21.15.39:0/2216557191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:17.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f23400640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f241072d0 0x7f9f24100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 -- 172.21.15.39:0/2216557191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f241072d0 msgr2=0x7f9f24100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f241072d0 0x7f9f24100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 -- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 msgr2=0x7f9f24101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 0x7f9f24101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.977+0000 7f9f1be00640 1 -- 172.21.15.39:0/2216557191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f24100e20 con 0x7f9f241049a0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f1be00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9c0 tx=0x7f9f0c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00ccc0 con 0x7f9f241049a0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a1a20 con 0x7f9f241049a0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f0c010910 con 0x7f9f241049a0 2024-03-16T10:08:17.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c018880 con 0x7f9f241049a0 2024-03-16T10:08:17.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.978+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a1f90 con 0x7f9f241049a0 2024-03-16T10:08:17.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.979+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f240fd550 con 0x7f9f241049a0 2024-03-16T10:08:17.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.981+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9f0c010ac0 con 0x7f9f241049a0 2024-03-16T10:08:17.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.982+0000 7f9f1b400640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 0x7f9f080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:17.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.983+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f9f0c0cf1d0 con 0x7f9f241049a0 2024-03-16T10:08:17.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.983+0000 7f9f22a00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 0x7f9f080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:17.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.984+0000 7f9f22a00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 0x7f9f080a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f1402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:17.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:17.986+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f0c09b810 con 0x7f9f241049a0 2024-03-16T10:08:18.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.193+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f240623e0 con 0x7f9f241049a0 2024-03-16T10:08:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.195+0000 7f9f1b400640 1 -- 172.21.15.39:0/2216557191 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24850 (secure 0 0 0) 0x7f9f0c09f8b0 con 0x7f9f241049a0 2024-03-16T10:08:18.197 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:18.197 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:08:14.004515+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:18.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.198+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 msgr2=0x7f9f080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.198+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 0x7f9f080a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f1402c880 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.198+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 msgr2=0x7f9f24100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.198+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9c0 tx=0x7f9f0c00ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.198+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 shutdown_connections 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f080a2800 0x7f9f080a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f241072d0 0x7f9f24100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f241049a0 0x7f9f24100020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 --2- 172.21.15.39:0/2216557191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f24102070 0x7f9f24101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 >> 172.21.15.39:0/2216557191 conn(0x7f9f240fba60 msgr2=0x7f9f24102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 shutdown_connections 2024-03-16T10:08:18.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.199+0000 7f9f23e00640 1 -- 172.21.15.39:0/2216557191 wait complete. 2024-03-16T10:08:18.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:08:18.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:18 smithi105 ceph-mon[36559]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:18.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:18 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2608053805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:18.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:18 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/506290882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 -- 172.21.15.39:0/225166682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e41029b0 msgr2=0x7f31e4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 --2- 172.21.15.39:0/225166682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e41029b0 0x7f31e4104e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009900 tx=0x7f31d002bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 -- 172.21.15.39:0/225166682 shutdown_connections 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 --2- 172.21.15.39:0/225166682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4105360 0x7f31e41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 --2- 172.21.15.39:0/225166682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e41029b0 0x7f31e4104e20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 --2- 172.21.15.39:0/225166682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e4100040 0x7f31e4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 -- 172.21.15.39:0/225166682 >> 172.21.15.39:0/225166682 conn(0x7f31e40fbc40 msgr2=0x7f31e40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 -- 172.21.15.39:0/225166682 shutdown_connections 2024-03-16T10:08:18.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.431+0000 7f31ec600640 1 -- 172.21.15.39:0/225166682 wait complete. 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.432+0000 7f31ec600640 1 Processor -- start 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.432+0000 7f31ec600640 1 -- start start 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4100040 0x7f31e419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e4105360 0x7f31e41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4113a90 con 0x7f31e4100040 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4113910 con 0x7f31e41029b0 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ec600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f31e4113c10 con 0x7f31e4105360 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31e9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31ea600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e4105360 0x7f31e41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31e9200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36164/0 (socket says 172.21.15.39:36164) 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31e9200640 1 -- 172.21.15.39:0/1073202147 learned_addr learned my addr 172.21.15.39:0/1073202147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.433+0000 7f31e9c00640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4100040 0x7f31e419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 -- 172.21.15.39:0/1073202147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e4105360 msgr2=0x7f31e41a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e4105360 0x7f31e41a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 -- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4100040 msgr2=0x7f31e419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4100040 0x7f31e419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 -- 172.21.15.39:0/1073202147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e41a5ce0 con 0x7f31e41029b0 2024-03-16T10:08:18.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e9200640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31d002c8f0 tx=0x7f31d0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0041070 con 0x7f31e41029b0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d00044d0 con 0x7f31e41029b0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e41a5f70 con 0x7f31e41029b0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0034440 con 0x7f31e41029b0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.434+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e41a63d0 con 0x7f31e41029b0 2024-03-16T10:08:18.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.436+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e40fd730 con 0x7f31e41029b0 2024-03-16T10:08:18.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.437+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f31d0044070 con 0x7f31e41029b0 2024-03-16T10:08:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.438+0000 7f31e2000640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 0x7f31cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.439+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f31d00f0970 con 0x7f31e41029b0 2024-03-16T10:08:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.439+0000 7f31e9c00640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 0x7f31cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.439+0000 7f31e9c00640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 0x7f31cc0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009bd0 tx=0x7f31d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.445+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f31d00bcff0 con 0x7f31e41029b0 2024-03-16T10:08:18.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:18 smithi194 ceph-mon[36445]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:18.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:18 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2608053805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:18.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:18 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/506290882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:18.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:18 smithi039 ceph-mon[31510]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:18.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:18 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2608053805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:18.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:18 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/506290882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:18.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.688+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e4062480 con 0x7f31e41029b0 2024-03-16T10:08:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.689+0000 7f31e2000640 1 -- 172.21.15.39:0/1073202147 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f31d00c1090 con 0x7f31e41029b0 2024-03-16T10:08:18.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:18.691 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:08:18.691 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.691+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 msgr2=0x7f31cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 0x7f31cc0a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009bd0 tx=0x7f31d8005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 msgr2=0x7f31e419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31d002c8f0 tx=0x7f31d0005d50 comp rx=0 tx=0).stop 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 shutdown_connections 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f31cc0a2800 0x7f31cc0a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31e4105360 0x7f31e41a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31e41029b0 0x7f31e419e650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 --2- 172.21.15.39:0/1073202147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e4100040 0x7f31e419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 >> 172.21.15.39:0/1073202147 conn(0x7f31e40fbc40 msgr2=0x7f31e41009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 shutdown_connections 2024-03-16T10:08:18.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.692+0000 7f31ec600640 1 -- 172.21.15.39:0/1073202147 wait complete. 2024-03-16T10:08:18.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T10:06:03.734324+0000', 'modified': '2024-03-16T10:06:40.587364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24544, 'mds_3': 24523, 'mds_4': 24518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/1279561047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 1279561047}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 1279561047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/1951962450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1951962450}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1951962450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/2605661802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 2605661802}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 2605661802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/106855660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 106855660}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 106855660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/3003167044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 3003167044}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 3003167044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.105:6837/3830303638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3830303638}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3830303638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.194:6837/3973444513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3973444513}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3973444513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.194:6839/985236448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 985236448}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 985236448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6841/1425412361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 1425412361}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 1425412361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6839/2189492991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 2189492991}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 2189492991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24557, 24569, 24518, 24523, 24544]} 2024-03-16T10:08:18.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-16T10:08:18.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T10:08:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 -- 172.21.15.39:0/3510601336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de81028a0 msgr2=0x7f6de8104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 --2- 172.21.15.39:0/3510601336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de81028a0 0x7f6de8104d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009900 tx=0x7f6dd402bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 -- 172.21.15.39:0/3510601336 shutdown_connections 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 --2- 172.21.15.39:0/3510601336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 0x7f6de81056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 --2- 172.21.15.39:0/3510601336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de81028a0 0x7f6de8104d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 --2- 172.21.15.39:0/3510601336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de80fff30 0x7f6de8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 -- 172.21.15.39:0/3510601336 >> 172.21.15.39:0/3510601336 conn(0x7f6de80fbb50 msgr2=0x7f6de80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 -- 172.21.15.39:0/3510601336 shutdown_connections 2024-03-16T10:08:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.927+0000 7f6ded200640 1 -- 172.21.15.39:0/3510601336 wait complete. 2024-03-16T10:08:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.928+0000 7f6ded200640 1 Processor -- start 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.928+0000 7f6ded200640 1 -- start start 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.928+0000 7f6ded200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.928+0000 7f6ded200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de81028a0 0x7f6de819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.928+0000 7f6ded200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 0x7f6de81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6ded200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8113960 con 0x7f6de81028a0 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6ded200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6de81137e0 con 0x7f6de80fff30 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6ded200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6de8113ae0 con 0x7f6de8105250 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de81028a0 0x7f6de819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36176/0 (socket says 172.21.15.39:36176) 2024-03-16T10:08:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 -- 172.21.15.39:0/2689753918 learned_addr learned my addr 172.21.15.39:0/2689753918 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7e00640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 0x7f6de81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 -- 172.21.15.39:0/2689753918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 msgr2=0x7f6de81a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 0x7f6de81a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 -- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de81028a0 msgr2=0x7f6de819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de81028a0 0x7f6de819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.929+0000 7f6de7400640 1 -- 172.21.15.39:0/2689753918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de81a5b70 con 0x7f6de80fff30 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6de7400640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e9f0 tx=0x7f6dd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6ddfe00640 1 -- 172.21.15.39:0/2689753918 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800cd80 con 0x7f6de80fff30 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6ddfe00640 1 -- 172.21.15.39:0/2689753918 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8010910 con 0x7f6de80fff30 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de81a5e60 con 0x7f6de80fff30 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6ddfe00640 1 -- 172.21.15.39:0/2689753918 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8018990 con 0x7f6de80fff30 2024-03-16T10:08:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.930+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de81a62c0 con 0x7f6de80fff30 2024-03-16T10:08:18.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.933+0000 7f6df146e740 1 Processor -- start 2024-03-16T10:08:18.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.933+0000 7f6df146e740 1 -- start start 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 0x561c96d10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10d30 0x561c96d15830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 0x561c96d18330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c96c74c50 con 0x561c96d10d30 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c96c1d9d0 con 0x561c96d10340 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6df146e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c96c70cf0 con 0x561c96d15ea0 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 0x561c96d10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 0x561c96d18330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 0x561c96d10760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36186/0 (socket says 172.21.15.39:36186) 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de7e00640 1 -- 172.21.15.39:0/587278173 learned_addr learned my addr 172.21.15.39:0/587278173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 -- 172.21.15.39:0/587278173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 msgr2=0x561c96d10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 0x561c96d10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 -- 172.21.15.39:0/587278173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10d30 msgr2=0x561c96d15830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10d30 0x561c96d15830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.934+0000 7f6de6a00640 1 -- 172.21.15.39:0/587278173 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c96c37f10 con 0x561c96d15ea0 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.935+0000 7f6ddfe00640 1 -- 172.21.15.39:0/2689753918 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6dd8010ac0 con 0x7f6de80fff30 2024-03-16T10:08:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.935+0000 7f6de6a00640 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 0x561c96d18330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036040 tx=0x7f6dd402c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53bd416a0d62cce server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.935+0000 7f6ddcc00640 1 -- 172.21.15.39:0/587278173 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4045070 con 0x561c96d15ea0 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.935+0000 7f6ddcc00640 1 -- 172.21.15.39:0/587278173 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4004040 con 0x561c96d15ea0 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.935+0000 7f6ddcc00640 1 -- 172.21.15.39:0/587278173 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4004eb0 con 0x561c96d15ea0 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- 172.21.15.39:0/587278173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 msgr2=0x561c96d18330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 0x561c96d18330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4036040 tx=0x7f6dd402c680 comp rx=0 tx=0).stop 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- 172.21.15.39:0/587278173 shutdown_connections 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d15ea0 0x561c96d18330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10d30 0x561c96d15830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 --2- 172.21.15.39:0/587278173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10340 0x561c96d10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- 172.21.15.39:0/587278173 >> 172.21.15.39:0/587278173 conn(0x561c96d0b8b0 msgr2=0x561c96d0bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- 172.21.15.39:0/587278173 shutdown_connections 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- 172.21.15.39:0/587278173 wait complete. 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6ddfe00640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 0x7f6dcc0a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.936+0000 7f6df146e740 1 -- start start 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6ddfe00640 1 -- 172.21.15.39:0/2689753918 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f6dd80cf1a0 con 0x7f6de80fff30 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6de7e00640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 0x7f6dcc0a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 Processor -- start 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d21a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c96d18df0 con 0x561c96d15ea0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c96c30e20 con 0x561c96d10d30 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c96d18f70 con 0x561c96d10340 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.937+0000 7f6df146e740 10 client.0 _ll_register_callbacks cb 0x561c96c307f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7e00640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 0x7f6dcc0a4bd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x561c96d11590 tx=0x7f6de0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54094/0 (socket says 172.21.15.39:54094) 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7400640 1 -- 172.21.15.39:0/1227764177 learned_addr learned my addr 172.21.15.39:0/1227764177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36192/0 (socket says 172.21.15.39:36192) 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7e00640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d21a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 -- 172.21.15.39:0/1227764177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 msgr2=0x561c96d1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 -- 172.21.15.39:0/1227764177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 msgr2=0x561c96d21a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d21a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 -- 172.21.15.39:0/1227764177 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c969892d0 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de7400640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd4004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5009b53a1dce5b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.938+0000 7f6dd1600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4045040 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd4004040 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f6dd4004eb0 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd4058050 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6df146e740 10 client.25145 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c96a536b0 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.939+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd4004040 con 0x561c96d10d30 2024-03-16T10:08:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd4003a60 con 0x561c96d10d30 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 10 client.25145 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 10 client.25145 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 20 client.25145 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d26e40 0x561c96d282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 4 client.25145 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 --> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c96cd22b0 con 0x561c96d26e40 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.940+0000 7f6de7400640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d26e40 0x561c96d282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.941+0000 7f6de7400640 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d26e40 0x561c96d282b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd4055df208d006c server_cookie=177a7154716359a9 in_seq=0 out_seq=0 2024-03-16T10:08:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.941+0000 7f6dd1600640 10 client.25145 ms_handle_connect on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6dd1600640 1 -- 172.21.15.39:0/1227764177 <== mds.4 v2:172.21.15.105:6836/3830303638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c96cd22b0 con 0x561c96d26e40 2024-03-16T10:08:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6dd1600640 10 client.25145 handle_command_reply: tid=0 2024-03-16T10:08:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 client.25145 shutdown 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 msgr2=0x561c96d1aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006010 tx=0x7f6dd4004420 comp rx=0 tx=0).stop 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 shutdown_connections 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d26e40 0x561c96d282b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d21a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d1aa10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 --2- 172.21.15.39:0/1227764177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6dd1600640 0 client.25145 ms_handle_reset on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 >> 172.21.15.39:0/1227764177 conn(0x561c96d0b8b0 msgr2=0x561c96d0aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 shutdown_connections 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 1 -- 172.21.15.39:0/1227764177 wait complete. 2024-03-16T10:08:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.942+0000 7f6df146e740 20 client.25145 trim_cache size 0 max 16384 2024-03-16T10:08:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 Processor -- start 2024-03-16T10:08:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 -- start start 2024-03-16T10:08:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d19310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c96d265f0 con 0x561c96d15ea0 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c96d26cc0 con 0x561c96d10d30 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6df146e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c96d27390 con 0x561c96d10340 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6de7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6de6a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6de7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54098/0 (socket says 172.21.15.39:54098) 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.963+0000 7f6de7400640 1 -- 172.21.15.39:0/3690956635 learned_addr learned my addr 172.21.15.39:0/3690956635 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7e00640 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d19310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 -- 172.21.15.39:0/3690956635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 msgr2=0x561c96d21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 -- 172.21.15.39:0/3690956635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 msgr2=0x561c96d19310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d19310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 -- 172.21.15.39:0/3690956635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c96d11110 con 0x561c96d10340 2024-03-16T10:08:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6de7400640 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80981b0 tx=0x7f6dd8023040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22c09fe09daa15f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6dd3e00640 1 -- 172.21.15.39:0/3690956635 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80cc9a0 con 0x561c96d10340 2024-03-16T10:08:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6dd3e00640 1 -- 172.21.15.39:0/3690956635 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80b4070 con 0x561c96d10340 2024-03-16T10:08:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.964+0000 7f6dd3e00640 1 -- 172.21.15.39:0/3690956635 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8018e20 con 0x561c96d10340 2024-03-16T10:08:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 -- 172.21.15.39:0/3690956635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 msgr2=0x561c96d0a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80981b0 tx=0x7f6dd8023040 comp rx=0 tx=0).stop 2024-03-16T10:08:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 -- 172.21.15.39:0/3690956635 shutdown_connections 2024-03-16T10:08:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d15ea0 0x561c96d19310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d10d30 0x561c96d21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 --2- 172.21.15.39:0/3690956635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10340 0x561c96d0a8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 -- 172.21.15.39:0/3690956635 >> 172.21.15.39:0/3690956635 conn(0x561c96d0b8b0 msgr2=0x561c96d0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 -- 172.21.15.39:0/3690956635 shutdown_connections 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.965+0000 7f6df146e740 1 -- 172.21.15.39:0/3690956635 wait complete. 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 -- start start 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 Processor -- start 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10340 0x561c96d1d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 0x561c96d1d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 0x561c96d205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561c96d28130 con 0x561c96d10340 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c96d27a60 con 0x561c96d15ea0 2024-03-16T10:08:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c96d28800 con 0x561c96d10d30 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6df146e740 10 client.0 _ll_register_callbacks cb 0x7f6dd40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6de7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 0x561c96d205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6de7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10340 0x561c96d1d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6de6a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 0x561c96d1d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6de7e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 0x561c96d205b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36206/0 (socket says 172.21.15.39:36206) 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.966+0000 7f6de7e00640 1 -- 172.21.15.39:0/1234649168 learned_addr learned my addr 172.21.15.39:0/1234649168 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 -- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 msgr2=0x561c96d205b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 0x561c96d205b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 -- 172.21.15.39:0/1234649168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10340 msgr2=0x561c96d1d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10340 0x561c96d1d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 -- 172.21.15.39:0/1234649168 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c96d20ba0 con 0x561c96d10d30 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6de6a00640 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 0x561c96d1d590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dd402c7b0 tx=0x7f6dd402c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2965bd3b073e91d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6dd1600640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4045040 con 0x561c96d10d30 2024-03-16T10:08:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f6dd404a450 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.967+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd80b4070 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6dd40035f0 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 10 client.25159 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c96d8cbb0 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80b4070 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6dd404aac0 con 0x561c96d10d30 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 10 client.25159 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 10 client.25159 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 20 client.25159 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d914f0 0x561c96d90200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 4 client.25159 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.968+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 --> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] -- command(tid 0: {"prefix": "status"}) -- 0x561c96c82190 con 0x561c96d914f0 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6de7400640 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d914f0 0x561c96d90200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6de7400640 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d914f0 0x561c96d90200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfcb259da4fec04 server_cookie=e06fb2c808c9be2a in_seq=0 out_seq=0 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6dd1600640 10 client.25159 ms_handle_connect on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6dd1600640 1 -- 172.21.15.39:0/1234649168 <== mds.4 v2:172.21.15.105:6836/3830303638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561c96c82190 con 0x561c96d914f0 2024-03-16T10:08:18.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6dd1600640 10 client.25159 handle_command_reply: tid=0 2024-03-16T10:08:18.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.969+0000 7f6df146e740 1 client.25159 shutdown 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 msgr2=0x561c96d1d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 0x561c96d1d590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6dd402c7b0 tx=0x7f6dd402c6d0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 shutdown_connections 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x561c96d914f0 0x561c96d90200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c96d15ea0 0x561c96d205b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c96d10d30 0x561c96d1d590 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 --2- 172.21.15.39:0/1234649168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561c96d10340 0x561c96d1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 >> 172.21.15.39:0/1234649168 conn(0x561c96d0b8b0 msgr2=0x561c96d0aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6dd1600640 0 client.25159 ms_handle_reset on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 shutdown_connections 2024-03-16T10:08:18.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 1 -- 172.21.15.39:0/1234649168 wait complete. 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.970+0000 7f6df146e740 20 client.25159 trim_cache size 0 max 16384 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24518, 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:18.976 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 99.410605908999997, 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 153.747012506 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 msgr2=0x7f6dcc0a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 0x7f6dcc0a4bd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x561c96d11590 tx=0x7f6de0009500 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 msgr2=0x7f6de819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e9f0 tx=0x7f6dd800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 shutdown_connections 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f6dcc0a2710 0x7f6dcc0a4bd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6de8105250 0x7f6de81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de81028a0 0x7f6de819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 --2- 172.21.15.39:0/2689753918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6de80fff30 0x7f6de819df50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 >> 172.21.15.39:0/2689753918 conn(0x7f6de80fbb50 msgr2=0x7f6de81008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 shutdown_connections 2024-03-16T10:08:18.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:18.971+0000 7f6ded200640 1 -- 172.21.15.39:0/2689753918 wait complete. 2024-03-16T10:08:18.989 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 -- 172.21.15.39:0/1220202351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d0102170 msgr2=0x7ff0d0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 --2- 172.21.15.39:0/1220202351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d0102170 0x7ff0d0108a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009ca0 tx=0x7ff0c802c100 comp rx=0 tx=0).stop 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 -- 172.21.15.39:0/1220202351 shutdown_connections 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 --2- 172.21.15.39:0/1220202351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d0102170 0x7ff0d0108a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 --2- 172.21.15.39:0/1220202351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d01017b0 0x7ff0d0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 --2- 172.21.15.39:0/1220202351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01005c0 0x7ff0d01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 -- 172.21.15.39:0/1220202351 >> 172.21.15.39:0/1220202351 conn(0x7ff0d00fbb50 msgr2=0x7ff0d00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 -- 172.21.15.39:0/1220202351 shutdown_connections 2024-03-16T10:08:19.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.197+0000 7ff0d5000640 1 -- 172.21.15.39:0/1220202351 wait complete. 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.198+0000 7ff0d5000640 1 Processor -- start 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 -- start start 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 0x7ff0d0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d0102170 0x7ff0d01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010c220 con 0x7ff0d0102170 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010c0a0 con 0x7ff0d01005c0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0d5000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010c3a0 con 0x7ff0d01017b0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0cf400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 0x7ff0d0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54116/0 (socket says 172.21.15.39:54116) 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0cea00640 1 -- 172.21.15.39:0/2178490741 learned_addr learned my addr 172.21.15.39:0/2178490741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.199+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d0102170 0x7ff0d01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 -- 172.21.15.39:0/2178490741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 msgr2=0x7ff0d019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 -- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d0102170 msgr2=0x7ff0d01a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d0102170 0x7ff0d01a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 -- 172.21.15.39:0/2178490741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a1750 con 0x7ff0d01005c0 2024-03-16T10:08:19.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cea00640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 0x7ff0d0199b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000e9c0 tx=0x7ff0c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0c7e00640 1 -- 172.21.15.39:0/2178490741 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000ccc0 con 0x7ff0d01005c0 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0c7e00640 1 -- 172.21.15.39:0/2178490741 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c00035f0 con 0x7ff0d01005c0 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0c7e00640 1 -- 172.21.15.39:0/2178490741 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000ce60 con 0x7ff0d01005c0 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.200+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a1a40 con 0x7ff0d01005c0 2024-03-16T10:08:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.201+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a1f90 con 0x7ff0d01005c0 2024-03-16T10:08:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 Processor -- start 2024-03-16T10:08:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 -- start start 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8ef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56147d84bc50 con 0x56147d8e7340 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56147d7f49d0 con 0x56147d8e7d30 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0d942c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56147d847cf0 con 0x56147d8ecea0 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0c7e00640 1 -- 172.21.15.39:0/2178490741 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff0c00037a0 con 0x7ff0d01005c0 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.204+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38942/0 (socket says 172.21.15.39:38942) 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cfe00640 1 -- 172.21.15.39:0/1310976783 learned_addr learned my addr 172.21.15.39:0/1310976783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 -- 172.21.15.39:0/1310976783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 msgr2=0x56147d8ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 -- 172.21.15.39:0/1310976783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 msgr2=0x56147d8e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 -- 172.21.15.39:0/1310976783 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147d80ef10 con 0x56147d8ecea0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.205+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8ef330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005fd0 tx=0x7ff0bc0109e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c97134cd28dc34d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0c4c00640 1 -- 172.21.15.39:0/1310976783 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011820 con 0x56147d8ecea0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0c4c00640 1 -- 172.21.15.39:0/1310976783 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc00e3b0 con 0x56147d8ecea0 2024-03-16T10:08:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0c4c00640 1 -- 172.21.15.39:0/1310976783 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00d5a0 con 0x56147d8ecea0 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0c7e00640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 0x7ff0b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0c7e00640 1 -- 172.21.15.39:0/2178490741 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7ff0c00cf1d0 con 0x7ff0d01005c0 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 0x7ff0b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0d942c740 1 -- 172.21.15.39:0/1310976783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 msgr2=0x56147d8ef330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.206+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8ef330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005fd0 tx=0x7ff0bc0109e0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 -- 172.21.15.39:0/1310976783 shutdown_connections 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8ef330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1310976783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 -- 172.21.15.39:0/1310976783 >> 172.21.15.39:0/1310976783 conn(0x56147d8e28b0 msgr2=0x56147d8e2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 -- 172.21.15.39:0/1310976783 shutdown_connections 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 -- 172.21.15.39:0/1310976783 wait complete. 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 0x7ff0b40a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8032420 tx=0x7ff0c8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.207+0000 7ff0d942c740 1 -- start start 2024-03-16T10:08:19.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 Processor -- start 2024-03-16T10:08:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56147d8efe50 con 0x56147d8e7d30 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56147d807e20 con 0x56147d8ecea0 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56147d8effd0 con 0x56147d8e7340 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0d942c740 10 client.0 _ll_register_callbacks cb 0x56147d8077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0cf400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36244/0 (socket says 172.21.15.39:36244) 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.208+0000 7ff0cfe00640 1 -- 172.21.15.39:0/1998123259 learned_addr learned my addr 172.21.15.39:0/1998123259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 -- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 msgr2=0x56147d8f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 -- 172.21.15.39:0/1998123259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 msgr2=0x56147d8f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 -- 172.21.15.39:0/1998123259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147d5602d0 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0c802c640 tx=0x7ff0c80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abaa9761eacaac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.209+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8046070 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7ff0c8004180 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0bc00e3b0 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0c805b050 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 10 client.25160 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56147d62a6b0 con 0x56147d8ecea0 2024-03-16T10:08:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc00e3b0 con 0x56147d8ecea0 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c804dc80 con 0x56147d8ecea0 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 10 client.25160 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 10 client.25160 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 20 client.25160 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.210+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d8fe280 0x56147d8ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.211+0000 7ff0d942c740 4 client.25160 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.211+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 --> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56147d8a92b0 con 0x56147d8fe280 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.211+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d8fe280 0x56147d8ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.211+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d8fe280 0x56147d8ff6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b155018dc27a850 server_cookie=177d1bd74f69ac8a in_seq=0 out_seq=0 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.211+0000 7ff0b9600640 10 client.25160 ms_handle_connect on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0b9600640 1 -- 172.21.15.39:0/1998123259 <== mds.3 v2:172.21.15.194:6836/3973444513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56147d8a92b0 con 0x56147d8fe280 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0b9600640 10 client.25160 handle_command_reply: tid=0 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 client.25160 shutdown 2024-03-16T10:08:19.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 msgr2=0x56147d8f8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff0c802c640 tx=0x7ff0c80095c0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 shutdown_connections 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d8fe280 0x56147d8ff6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0b9600640 0 client.25160 ms_handle_reset on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8ecea0 0x56147d8f8a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1998123259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.212+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 >> 172.21.15.39:0/1998123259 conn(0x56147d8e28b0 msgr2=0x56147d8e1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.213+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 shutdown_connections 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.213+0000 7ff0d942c740 1 -- 172.21.15.39:0/1998123259 wait complete. 2024-03-16T10:08:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.213+0000 7ff0d942c740 20 client.25160 trim_cache size 0 max 16384 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 Processor -- start 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 -- start start 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56147d8fda30 con 0x56147d8e7340 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56147d8fe100 con 0x56147d8e7d30 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.246+0000 7ff0d942c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56147d8fe7d0 con 0x56147d8ecea0 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36250/0 (socket says 172.21.15.39:36250) 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 -- 172.21.15.39:0/1364229489 learned_addr learned my addr 172.21.15.39:0/1364229489 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cf400640 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 -- 172.21.15.39:0/1364229489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 msgr2=0x56147d8f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 -- 172.21.15.39:0/1364229489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 msgr2=0x56147d8f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.247+0000 7ff0cea00640 1 -- 172.21.15.39:0/1364229489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147d8e3620 con 0x56147d8e7d30 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.248+0000 7ff0cea00640 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc01a000 tx=0x7ff0bc0104c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd477f11ccb3c08c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.248+0000 7ff0bbe00640 1 -- 172.21.15.39:0/1364229489 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc02b070 con 0x56147d8e7d30 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.248+0000 7ff0bbe00640 1 -- 172.21.15.39:0/1364229489 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc004680 con 0x56147d8e7d30 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.248+0000 7ff0bbe00640 1 -- 172.21.15.39:0/1364229489 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011930 con 0x56147d8e7d30 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- 172.21.15.39:0/1364229489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 msgr2=0x56147d8f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc01a000 tx=0x7ff0bc0104c0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- 172.21.15.39:0/1364229489 shutdown_connections 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8ecea0 0x56147d8f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7d30 0x56147d8f1a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 --2- 172.21.15.39:0/1364229489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8e7340 0x56147d8f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- 172.21.15.39:0/1364229489 >> 172.21.15.39:0/1364229489 conn(0x56147d8e28b0 msgr2=0x56147d8dd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- 172.21.15.39:0/1364229489 shutdown_connections 2024-03-16T10:08:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- 172.21.15.39:0/1364229489 wait complete. 2024-03-16T10:08:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 -- start start 2024-03-16T10:08:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.249+0000 7ff0d942c740 1 Processor -- start 2024-03-16T10:08:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7340 0x56147d8f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8ecea0 0x56147d8f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56147d8ff570 con 0x56147d8ecea0 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56147d8feea0 con 0x56147d8e7340 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56147d8ffc40 con 0x56147d8e7d30 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0d942c740 10 client.0 _ll_register_callbacks cb 0x7ff0c8032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54146/0 (socket says 172.21.15.39:54146) 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0cea00640 1 -- 172.21.15.39:0/2828224312 learned_addr learned my addr 172.21.15.39:0/2828224312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.250+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7340 0x56147d8f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cfe00640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8ecea0 0x56147d8f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 -- 172.21.15.39:0/2828224312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7340 msgr2=0x56147d8f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7340 0x56147d8f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 -- 172.21.15.39:0/2828224312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8ecea0 msgr2=0x56147d8f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8ecea0 0x56147d8f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 -- 172.21.15.39:0/2828224312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147d8f7990 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0cea00640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00be40 tx=0x7ff0bc0052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09cbdc1226d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc034060 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0bc004680 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7ff0bc011690 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0bc03b050 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0d942c740 10 client.25177 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.251+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56147d8f8c90 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0bc004680 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0bc02e030 con 0x56147d8e7d30 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 10 client.25177 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 10 client.25177 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 20 client.25177 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d968100 0x56147d966e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 4 client.25177 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 --> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] -- command(tid 0: {"prefix": "status"}) -- 0x56147d859190 con 0x56147d968100 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.252+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d968100 0x56147d966e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0cf400640 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d968100 0x56147d966e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed41e111cb3b0ab5 server_cookie=9d4b22076a24198e in_seq=0 out_seq=0 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0b9600640 10 client.25177 ms_handle_connect on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0b9600640 1 -- 172.21.15.39:0/2828224312 <== mds.3 v2:172.21.15.194:6836/3973444513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56147d859190 con 0x56147d968100 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0b9600640 10 client.25177 handle_command_reply: tid=0 2024-03-16T10:08:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 client.25177 shutdown 2024-03-16T10:08:19.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:19 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2216557191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:19 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1073202147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 msgr2=0x56147d8f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00be40 tx=0x7ff0bc0052f0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 shutdown_connections 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x56147d968100 0x56147d966e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56147d8ecea0 0x56147d8f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56147d8e7d30 0x56147d8f4380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 --2- 172.21.15.39:0/2828224312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56147d8e7340 0x56147d8f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0b9600640 0 client.25177 ms_handle_reset on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.253+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 >> 172.21.15.39:0/2828224312 conn(0x56147d8e28b0 msgr2=0x56147d8f14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 shutdown_connections 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d942c740 1 -- 172.21.15.39:0/2828224312 wait complete. 2024-03-16T10:08:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d942c740 20 client.25177 trim_cache size 0 max 16384 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24523, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 100.711616353, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 156.809910296 2024-03-16T10:08:19.260 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 msgr2=0x7ff0b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 0x7ff0b40a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8032420 tx=0x7ff0c8032590 comp rx=0 tx=0).stop 2024-03-16T10:08:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 msgr2=0x7ff0d0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.254+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 0x7ff0d0199b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000e9c0 tx=0x7ff0c000ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 shutdown_connections 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0b40a2800 0x7ff0b40a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0d0102170 0x7ff0d01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d01017b0 0x7ff0d019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 --2- 172.21.15.39:0/2178490741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0d01005c0 0x7ff0d0199b30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 >> 172.21.15.39:0/2178490741 conn(0x7ff0d00fbb50 msgr2=0x7ff0d00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 shutdown_connections 2024-03-16T10:08:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.255+0000 7ff0d5000640 1 -- 172.21.15.39:0/2178490741 wait complete. 2024-03-16T10:08:19.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T10:08:19.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:19 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2216557191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:19 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1073202147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:19 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2216557191' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:19 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1073202147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 -- 172.21.15.39:0/974423248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 msgr2=0x7f880c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 --2- 172.21.15.39:0/974423248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c10be60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ca0 tx=0x7f880402c0f0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 -- 172.21.15.39:0/974423248 shutdown_connections 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 --2- 172.21.15.39:0/974423248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c10be60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 --2- 172.21.15.39:0/974423248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 --2- 172.21.15.39:0/974423248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 0x7f880c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 -- 172.21.15.39:0/974423248 >> 172.21.15.39:0/974423248 conn(0x7f880c0fef90 msgr2=0x7f880c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 -- 172.21.15.39:0/974423248 shutdown_connections 2024-03-16T10:08:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.504+0000 7f8811800640 1 -- 172.21.15.39:0/974423248 wait complete. 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.505+0000 7f8811800640 1 Processor -- start 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 -- start start 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 0x7f880c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10f680 con 0x7f880c103a20 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10f500 con 0x7f880c104c10 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f8811800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10f800 con 0x7f880c1055d0 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f880b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f880a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f880b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54172/0 (socket says 172.21.15.39:54172) 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f880b400640 1 -- 172.21.15.39:0/3754145286 learned_addr learned my addr 172.21.15.39:0/3754145286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.506+0000 7f880aa00640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 0x7f880c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 -- 172.21.15.39:0/3754145286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 msgr2=0x7f880c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 -- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 msgr2=0x7f880c199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 0x7f880c199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 -- 172.21.15.39:0/3754145286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1a1920 con 0x7f880c1055d0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880a000640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f880b400640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f880402c630 tx=0x7f8804032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f8803400640 1 -- 172.21.15.39:0/3754145286 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804041020 con 0x7f880c1055d0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f8803400640 1 -- 172.21.15.39:0/3754145286 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804034cd0 con 0x7f880c1055d0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.507+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1a1bb0 con 0x7f880c1055d0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.508+0000 7f8803400640 1 -- 172.21.15.39:0/3754145286 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804039040 con 0x7f880c1055d0 2024-03-16T10:08:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.508+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a20a0 con 0x7f880c1055d0 2024-03-16T10:08:19.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f8803400640 1 -- 172.21.15.39:0/3754145286 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8804044020 con 0x7f880c1055d0 2024-03-16T10:08:19.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 Processor -- start 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 -- start start 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f103ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103cacc0 0x558f103cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f10329c50 con 0x558f103c5b50 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f102d29d0 con 0x558f103cacc0 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.511+0000 7f88145a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f10325cf0 con 0x558f103c5160 2024-03-16T10:08:19.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54188/0 (socket says 172.21.15.39:54188) 2024-03-16T10:08:19.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 -- 172.21.15.39:0/505141911 learned_addr learned my addr 172.21.15.39:0/505141911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880aa00640 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f103ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103cacc0 0x558f103cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 -- 172.21.15.39:0/505141911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103cacc0 msgr2=0x558f103cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103cacc0 0x558f103cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 -- 172.21.15.39:0/505141911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 msgr2=0x558f103ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f103ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f880a000640 1 -- 172.21.15.39:0/505141911 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f102ecf10 con 0x558f103c5160 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.512+0000 7f8803400640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 0x7f87f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f880a000640 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87f8011c40 tx=0x7f87f8009bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21aa4493dc51be9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f8803400640 1 -- 172.21.15.39:0/3754145286 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f88040f0dd0 con 0x7f880c1055d0 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f880aa00640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 0x7f87f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f87f7e00640 1 -- 172.21.15.39:0/505141911 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800fc50 con 0x558f103c5160 2024-03-16T10:08:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f87f7e00640 1 -- 172.21.15.39:0/505141911 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f800fdf0 con 0x558f103c5160 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.513+0000 7f87f7e00640 1 -- 172.21.15.39:0/505141911 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800e710 con 0x558f103c5160 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- 172.21.15.39:0/505141911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 msgr2=0x558f103c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f87f8011c40 tx=0x7f87f8009bc0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f880aa00640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 0x7f87f00a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x558f103c5f50 tx=0x7f87fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- 172.21.15.39:0/505141911 shutdown_connections 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103cacc0 0x558f103cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f103ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 --2- 172.21.15.39:0/505141911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5160 0x558f103c5580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- 172.21.15.39:0/505141911 >> 172.21.15.39:0/505141911 conn(0x558f103c06d0 msgr2=0x558f103c0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- 172.21.15.39:0/505141911 shutdown_connections 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- 172.21.15.39:0/505141911 wait complete. 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.514+0000 7f88145a6740 1 -- start start 2024-03-16T10:08:19.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 Processor -- start 2024-03-16T10:08:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f103cdc70 con 0x558f103cacc0 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f102e5e20 con 0x558f103c5160 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f103cddf0 con 0x558f103c5b50 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f88145a6740 10 client.0 _ll_register_callbacks cb 0x558f102e57f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f880aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54192/0 (socket says 172.21.15.39:54192) 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.515+0000 7f880a000640 1 -- 172.21.15.39:0/2074109572 learned_addr learned my addr 172.21.15.39:0/2074109572 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880b400640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 -- 172.21.15.39:0/2074109572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 msgr2=0x558f103cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 -- 172.21.15.39:0/2074109572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 msgr2=0x558f103d68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103d68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 -- 172.21.15.39:0/2074109572 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1003e2d0 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880b400640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f880a000640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87f8005430 tx=0x7f87f8005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e5e13114b0ef6b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f87f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8025050 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.516+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f800fdf0 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f87f8004020 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f8034050 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 10 client.25189 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f101086b0 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f800fdf0 con 0x558f103c5b50 2024-03-16T10:08:19.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87f801d020 con 0x558f103c5b50 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 10 client.25189 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 10 client.25189 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 20 client.25189 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f103dbe80 0x558f103dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 4 client.25189 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.517+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 --> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f103872b0 con 0x558f103dbe80 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.518+0000 7f880aa00640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f103dbe80 0x558f103dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.518+0000 7f880aa00640 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f103dbe80 0x558f103dd2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65bfe1d49a08b58a server_cookie=9b19421ac2241c7 in_seq=0 out_seq=0 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.518+0000 7f87f4c00640 10 client.25189 ms_handle_connect on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f87f4c00640 1 -- 172.21.15.39:0/2074109572 <== mds.2 v2:172.21.15.194:6838/985236448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558f103872b0 con 0x558f103dbe80 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f87f4c00640 10 client.25189 handle_command_reply: tid=0 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 client.25189 shutdown 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 msgr2=0x558f103cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87f8005430 tx=0x7f87f8005460 comp rx=0 tx=0).stop 2024-03-16T10:08:19.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 shutdown_connections 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f103dbe80 0x558f103dd2f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103cf860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 --2- 172.21.15.39:0/2074109572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f87f4c00640 0 client.25189 ms_handle_reset on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.519+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 >> 172.21.15.39:0/2074109572 conn(0x558f103c06d0 msgr2=0x558f103bfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.520+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 shutdown_connections 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.520+0000 7f88145a6740 1 -- 172.21.15.39:0/2074109572 wait complete. 2024-03-16T10:08:19.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.520+0000 7f88145a6740 20 client.25189 trim_cache size 0 max 16384 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.540+0000 7f88145a6740 1 Processor -- start 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.540+0000 7f88145a6740 1 -- start start 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f103db630 con 0x558f103cacc0 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f103dbd00 con 0x558f103c5160 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f88145a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f103dc3d0 con 0x558f103c5b50 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36306/0 (socket says 172.21.15.39:36306) 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 -- 172.21.15.39:0/4048501747 learned_addr learned my addr 172.21.15.39:0/4048501747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880a000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54206/0 (socket says 172.21.15.39:54206) 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 -- 172.21.15.39:0/4048501747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 msgr2=0x558f103d6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 -- 172.21.15.39:0/4048501747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 msgr2=0x558f103c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880aa00640 1 -- 172.21.15.39:0/4048501747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f10396750 con 0x558f103c5160 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.541+0000 7f880a000640 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.542+0000 7f880aa00640 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x558f103c65c0 tx=0x7f87fc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898af242dc24a8ff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.542+0000 7f87f7400640 1 -- 172.21.15.39:0/4048501747 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc02cdc0 con 0x558f103c5160 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.542+0000 7f87f7400640 1 -- 172.21.15.39:0/4048501747 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc032070 con 0x558f103c5160 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.542+0000 7f87f7400640 1 -- 172.21.15.39:0/4048501747 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc021410 con 0x558f103c5160 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- 172.21.15.39:0/4048501747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 msgr2=0x558f103cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x558f103c65c0 tx=0x7f87fc017000 comp rx=0 tx=0).stop 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- 172.21.15.39:0/4048501747 shutdown_connections 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103cacc0 0x558f103c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103c5b50 0x558f103d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 --2- 172.21.15.39:0/4048501747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103cd0b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- 172.21.15.39:0/4048501747 >> 172.21.15.39:0/4048501747 conn(0x558f103c06d0 msgr2=0x558f103c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- 172.21.15.39:0/4048501747 shutdown_connections 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- 172.21.15.39:0/4048501747 wait complete. 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 -- start start 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.543+0000 7f88145a6740 1 Processor -- start 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f1043dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 0x558f104401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558f103dd170 con 0x558f103c5b50 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f103dcaa0 con 0x558f103c5160 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558f103dd840 con 0x558f103cacc0 2024-03-16T10:08:19.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f88145a6740 10 client.0 _ll_register_callbacks cb 0x7f87f8011b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 0x558f104401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103d4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36312/0 (socket says 172.21.15.39:36312) 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880aa00640 1 -- 172.21.15.39:0/4137489341 learned_addr learned my addr 172.21.15.39:0/4137489341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880a000640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f1043dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 -- 172.21.15.39:0/4137489341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 msgr2=0x558f103d4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103d4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 -- 172.21.15.39:0/4137489341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 msgr2=0x558f1043dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f1043dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.544+0000 7f880b400640 1 -- 172.21.15.39:0/4137489341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f10440a00 con 0x558f103cacc0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f880b400640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 0x558f104401d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88040ee4c0 tx=0x7f88040bcf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97207cf94f8cf9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f87f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040dbe60 con 0x558f103cacc0 2024-03-16T10:08:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f88040c10d0 con 0x558f103cacc0 2024-03-16T10:08:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87fc032070 con 0x558f103cacc0 2024-03-16T10:08:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88040d6200 con 0x558f103cacc0 2024-03-16T10:08:19.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f88145a6740 10 client.25195 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f10441d00 con 0x558f103cacc0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.545+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc032070 con 0x558f103cacc0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f880403c030 con 0x558f103cacc0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 10 client.25195 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 10 client.25195 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 20 client.25195 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f10446950 0x558f10445660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 4 client.25195 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 --> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] -- command(tid 0: {"prefix": "status"}) -- 0x558f10337190 con 0x558f10446950 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f880aa00640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f10446950 0x558f10445660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f880aa00640 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f10446950 0x558f10445660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fd0d48c0f21b867 server_cookie=72ae80e0f26621eb in_seq=0 out_seq=0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.546+0000 7f87f4c00640 10 client.25195 ms_handle_connect on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f87f4c00640 1 -- 172.21.15.39:0/4137489341 <== mds.2 v2:172.21.15.194:6838/985236448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558f10337190 con 0x558f10446950 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f87f4c00640 10 client.25195 handle_command_reply: tid=0 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 client.25195 shutdown 2024-03-16T10:08:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 msgr2=0x558f104401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 0x558f104401d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88040ee4c0 tx=0x7f88040bcf60 comp rx=0 tx=0).stop 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 shutdown_connections 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x558f10446950 0x558f10445660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558f103cacc0 0x558f104401d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558f103c5b50 0x558f1043dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 --2- 172.21.15.39:0/4137489341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f103c5160 0x558f103d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 >> 172.21.15.39:0/4137489341 conn(0x558f103c06d0 msgr2=0x558f103c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f87f4c00640 0 client.25195 ms_handle_reset on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 shutdown_connections 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 1 -- 172.21.15.39:0/4137489341 wait complete. 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.547+0000 7f88145a6740 20 client.25195 trim_cache size 0 max 16384 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:19.552 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24544, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 102.019985127, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 151.45769380300001 2024-03-16T10:08:19.553 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 msgr2=0x7f87f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 0x7f87f00a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x558f103c5f50 tx=0x7f87fc005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 msgr2=0x7f880c1a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f880402c630 tx=0x7f8804032ea0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 shutdown_connections 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f87f00a2800 0x7f87f00a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c1055d0 0x7f880c1a1280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880c104c10 0x7f880c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 --2- 172.21.15.39:0/3754145286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c103a20 0x7f880c199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 >> 172.21.15.39:0/3754145286 conn(0x7f880c0fef90 msgr2=0x7f880c107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 shutdown_connections 2024-03-16T10:08:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.548+0000 7f8811800640 1 -- 172.21.15.39:0/3754145286 wait complete. 2024-03-16T10:08:19.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-16T10:08:19.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 -- 172.21.15.39:0/3938393445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41017f0 msgr2=0x7f62c4101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 --2- 172.21.15.39:0/3938393445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41017f0 0x7f62c4101c10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009900 tx=0x7f62b802bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:19.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 -- 172.21.15.39:0/3938393445 shutdown_connections 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 --2- 172.21.15.39:0/3938393445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41033a0 0x7f62c4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 --2- 172.21.15.39:0/3938393445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 0x7f62c4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 --2- 172.21.15.39:0/3938393445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41017f0 0x7f62c4101c10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 -- 172.21.15.39:0/3938393445 >> 172.21.15.39:0/3938393445 conn(0x7f62c40fcd80 msgr2=0x7f62c40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 -- 172.21.15.39:0/3938393445 shutdown_connections 2024-03-16T10:08:19.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.761+0000 7f62cb400640 1 -- 172.21.15.39:0/3938393445 wait complete. 2024-03-16T10:08:19.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.763+0000 7f62cb400640 1 Processor -- start 2024-03-16T10:08:19.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.763+0000 7f62cb400640 1 -- start start 2024-03-16T10:08:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.763+0000 7f62cb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 0x7f62c4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62cb400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 0x7f62c41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62cb400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 0x7f62c419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62cb400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62c410d440 con 0x7f62c41017f0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62cb400640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f62c410d2c0 con 0x7f62c41029e0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62cb400640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62c410d5c0 con 0x7f62c41033a0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 0x7f62c4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 0x7f62c419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 0x7f62c4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39054/0 (socket says 172.21.15.39:39054) 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c8a00640 1 -- 172.21.15.39:0/3973589325 learned_addr learned my addr 172.21.15.39:0/3973589325 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c3e00640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 0x7f62c41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 -- 172.21.15.39:0/3973589325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 msgr2=0x7f62c41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 0x7f62c41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 -- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 msgr2=0x7f62c4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 0x7f62c4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.764+0000 7f62c9400640 1 -- 172.21.15.39:0/3973589325 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c419f780 con 0x7f62c41033a0 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62c9400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 0x7f62c419f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e940 tx=0x7f62bc00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62c1600640 1 -- 172.21.15.39:0/3973589325 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004d20 con 0x7f62c41033a0 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62c1600640 1 -- 172.21.15.39:0/3973589325 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc005020 con 0x7f62c41033a0 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62c1600640 1 -- 172.21.15.39:0/3973589325 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0116b0 con 0x7f62c41033a0 2024-03-16T10:08:19.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c419fa70 con 0x7f62c41033a0 2024-03-16T10:08:19.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.765+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c419ffc0 con 0x7f62c41033a0 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.769+0000 7f62c1600640 1 -- 172.21.15.39:0/3973589325 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f62bc0054d0 con 0x7f62c41033a0 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.769+0000 7f62ce1d0740 1 Processor -- start 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.769+0000 7f62ce1d0740 1 -- start start 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac08c30 0x5601bac15aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601bab69c50 con 0x5601bac05ad0 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601bab129d0 con 0x5601bac050e0 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5601bab65cf0 con 0x5601bac08c30 2024-03-16T10:08:19.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36336/0 (socket says 172.21.15.39:36336) 2024-03-16T10:08:19.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 -- 172.21.15.39:0/2545186569 learned_addr learned my addr 172.21.15.39:0/2545186569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac08c30 0x5601bac15aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 -- 172.21.15.39:0/2545186569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac08c30 msgr2=0x5601bac15aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac08c30 0x5601bac15aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 -- 172.21.15.39:0/2545186569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 msgr2=0x5601bac085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.770+0000 7f62c3e00640 1 -- 172.21.15.39:0/2545186569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601bab2cf10 con 0x5601bac050e0 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62b00093e0 tx=0x7f62b0009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72a203ac0aeef40 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62b5c00640 1 -- 172.21.15.39:0/2545186569 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000f7c0 con 0x5601bac050e0 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62b5c00640 1 -- 172.21.15.39:0/2545186569 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b000fe40 con 0x5601bac050e0 2024-03-16T10:08:19.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62b5c00640 1 -- 172.21.15.39:0/2545186569 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000e3b0 con 0x5601bac050e0 2024-03-16T10:08:19.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2545186569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 msgr2=0x5601bac05500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.771+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62b00093e0 tx=0x7f62b0009370 comp rx=0 tx=0).stop 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2545186569 shutdown_connections 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac08c30 0x5601bac15aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2545186569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac05500 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2545186569 >> 172.21.15.39:0/2545186569 conn(0x5601bac006d0 msgr2=0x5601bac00ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2545186569 shutdown_connections 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2545186569 wait complete. 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62c1600640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 0x7f62ac0a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62c1600640 1 -- 172.21.15.39:0/3973589325 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f62bc0cee40 con 0x7f62c41033a0 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62c8a00640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 0x7f62ac0a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.772+0000 7f62ce1d0740 1 -- start start 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 Processor -- start 2024-03-16T10:08:19.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62c8a00640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 0x7f62ac0a4c50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62b802c8e0 tx=0x7f62b802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac08c30 0x5601bac1f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601bac164a0 con 0x5601bac08c30 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601bab25e20 con 0x5601bac050e0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5601bac16620 con 0x5601bac05ad0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62ce1d0740 10 client.0 _ll_register_callbacks cb 0x5601bab257f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62c3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.773+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36346/0 (socket says 172.21.15.39:36346) 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 -- 172.21.15.39:0/9675544 learned_addr learned my addr 172.21.15.39:0/9675544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c9400640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac08c30 0x5601bac1f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 -- 172.21.15.39:0/9675544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 msgr2=0x5601bac17fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 -- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac08c30 msgr2=0x5601bac1f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac08c30 0x5601bac1f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 -- 172.21.15.39:0/9675544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601ba87e2d0 con 0x5601bac050e0 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c3e00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62c8a00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009230 tx=0x7f62b802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf94713e5719c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.774+0000 7f62aa000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b804f070 con 0x5601bac050e0 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f62b8004460 con 0x5601bac050e0 2024-03-16T10:08:19.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b000fe40 con 0x5601bac050e0 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f62b805b050 con 0x5601bac050e0 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62ce1d0740 10 client.25196 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601ba9486b0 con 0x5601bac050e0 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b000fe40 con 0x5601bac050e0 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f62b8003a60 con 0x5601bac050e0 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.775+0000 7f62ce1d0740 10 client.25196 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62ce1d0740 10 client.25196 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62ce1d0740 20 client.25196 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac248d0 0x5601bac25d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62ce1d0740 4 client.25196 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:19.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 --> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601babc72b0 con 0x5601bac248d0 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.776+0000 7f62c3e00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac248d0 0x5601bac25d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.777+0000 7f62c3e00640 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac248d0 0x5601bac25d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=623f8f52ecb8525e server_cookie=bf296dd0f52e82b0 in_seq=0 out_seq=0 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.777+0000 7f62aa000640 10 client.25196 ms_handle_connect on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.777+0000 7f62aa000640 1 -- 172.21.15.39:0/9675544 <== mds.1 v2:172.21.15.39:6840/1425412361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5601babc72b0 con 0x5601bac248d0 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.777+0000 7f62aa000640 10 client.25196 handle_command_reply: tid=0 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.777+0000 7f62ce1d0740 1 client.25196 shutdown 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 msgr2=0x5601bac17aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009230 tx=0x7f62b802bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 shutdown_connections 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac08c30 0x5601bac1f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac248d0 0x5601bac25d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/9675544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac17aa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62aa000640 0 client.25196 ms_handle_reset on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 >> 172.21.15.39:0/9675544 conn(0x5601bac006d0 msgr2=0x5601babffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 shutdown_connections 2024-03-16T10:08:19.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 1 -- 172.21.15.39:0/9675544 wait complete. 2024-03-16T10:08:19.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.778+0000 7f62ce1d0740 20 client.25196 trim_cache size 0 max 16384 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 Processor -- start 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 -- start start 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 0x5601bac08550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601bac24080 con 0x5601bac006d0 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601bac24750 con 0x5601bac050e0 2024-03-16T10:08:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5601bac24e20 con 0x5601bac05ad0 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62c9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62c9400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54280/0 (socket says 172.21.15.39:54280) 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62c9400640 1 -- 172.21.15.39:0/2541554857 learned_addr learned my addr 172.21.15.39:0/2541554857 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62c3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.799+0000 7f62c8a00640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 0x5601bac08550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 -- 172.21.15.39:0/2541554857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 msgr2=0x5601bac1efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 -- 172.21.15.39:0/2541554857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 msgr2=0x5601bac08550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 0x5601bac08550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 -- 172.21.15.39:0/2541554857 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601babd6750 con 0x5601bac05ad0 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c8a00640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 0x5601bac08550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62c9400640 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0cc530 tx=0x7f62bc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8236797503e84bee server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62b5200640 1 -- 172.21.15.39:0/2541554857 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc01f040 con 0x5601bac05ad0 2024-03-16T10:08:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62b5200640 1 -- 172.21.15.39:0/2541554857 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc0b4070 con 0x5601bac05ad0 2024-03-16T10:08:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.800+0000 7f62b5200640 1 -- 172.21.15.39:0/2541554857 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc09f0d0 con 0x5601bac05ad0 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2541554857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 msgr2=0x5601bac17dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f62bc0cc530 tx=0x7f62bc005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2541554857 shutdown_connections 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac05ad0 0x5601bac17dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2541554857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac006d0 0x5601bac08550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2541554857 >> 172.21.15.39:0/2541554857 conn(0x5601babfbd10 msgr2=0x5601bac176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2541554857 shutdown_connections 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.801+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2541554857 wait complete. 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 -- start start 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 Processor -- start 2024-03-16T10:08:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac1dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601bac25bc0 con 0x5601bac05ad0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601bac254f0 con 0x5601bac050e0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5601bac26290 con 0x5601bac006d0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.802+0000 7f62ce1d0740 10 client.0 _ll_register_callbacks cb 0x7f62b80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:54284/0 (socket says 172.21.15.39:54284) 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 -- 172.21.15.39:0/2241507737 learned_addr learned my addr 172.21.15.39:0/2241507737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c9400640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac1dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 -- 172.21.15.39:0/2241507737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 msgr2=0x5601bac1d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 -- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 msgr2=0x5601bac1dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac1dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 -- 172.21.15.39:0/2241507737 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601bac1e470 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c9400640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62c8a00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62b8036040 tx=0x7f62b804b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1048d51bfb6237 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.803+0000 7f62aa000640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8065070 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62bc0b4070 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f62b804b8e0 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f62b804f040 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62ce1d0740 10 client.25213 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5601bac8a7c0 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc0b4070 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.804+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f62b8068020 con 0x5601bac006d0 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 10 client.25213 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 10 client.25213 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 20 client.25213 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac08c30 0x5601bac8e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 4 client.25213 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:19.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 --> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] -- command(tid 0: {"prefix": "status"}) -- 0x5601bab77190 con 0x5601bac08c30 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.805+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac08c30 0x5601bac8e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62c3e00640 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac08c30 0x5601bac8e0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=485b6f973489d777 server_cookie=539d9749cff9f5c2 in_seq=0 out_seq=0 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62aa000640 10 client.25213 ms_handle_connect on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62aa000640 1 -- 172.21.15.39:0/2241507737 <== mds.1 v2:172.21.15.39:6840/1425412361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5601bab77190 con 0x5601bac08c30 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62aa000640 10 client.25213 handle_command_reply: tid=0 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 client.25213 shutdown 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 msgr2=0x5601bac1d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62b8036040 tx=0x7f62b804b6c0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 shutdown_connections 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601bac05ad0 0x5601bac1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601bac050e0 0x5601bac1d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5601bac08c30 0x5601bac8e0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 --2- 172.21.15.39:0/2241507737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5601bac006d0 0x5601bac1d180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 >> 172.21.15.39:0/2241507737 conn(0x5601babfbd10 msgr2=0x5601bac03930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62aa000640 0 client.25213 ms_handle_reset on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 shutdown_connections 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.806+0000 7f62ce1d0740 1 -- 172.21.15.39:0/2241507737 wait complete. 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62ce1d0740 20 client.25213 trim_cache size 0 max 16384 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24557, 2024-03-16T10:08:19.811 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 103.303341488, 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 143.590756521 2024-03-16T10:08:19.812 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 msgr2=0x7f62ac0a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 0x7f62ac0a4c50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62b802c8e0 tx=0x7f62b802e040 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 msgr2=0x7f62c419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 0x7f62c419f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e940 tx=0x7f62bc00ee00 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 shutdown_connections 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f62ac0a2790 0x7f62ac0a4c50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62c41033a0 0x7f62c419f110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c41029e0 0x7f62c41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 --2- 172.21.15.39:0/3973589325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62c41017f0 0x7f62c4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 >> 172.21.15.39:0/3973589325 conn(0x7f62c40fcd80 msgr2=0x7f62c4105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 shutdown_connections 2024-03-16T10:08:19.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:19.807+0000 7f62cb400640 1 -- 172.21.15.39:0/3973589325 wait complete. 2024-03-16T10:08:19.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T10:08:20.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 -- 172.21.15.39:0/763271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 msgr2=0x7fbc10101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 --2- 172.21.15.39:0/763271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10101b00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002bc50 comp rx=0 tx=0).stop 2024-03-16T10:08:20.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 -- 172.21.15.39:0/763271081 shutdown_connections 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 --2- 172.21.15.39:0/763271081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 0x7fbc10109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 --2- 172.21.15.39:0/763271081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc10102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 --2- 172.21.15.39:0/763271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10101b00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 -- 172.21.15.39:0/763271081 >> 172.21.15.39:0/763271081 conn(0x7fbc100fcc50 msgr2=0x7fbc100ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 -- 172.21.15.39:0/763271081 shutdown_connections 2024-03-16T10:08:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.039+0000 7fbc0f200640 1 -- 172.21.15.39:0/763271081 wait complete. 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 Processor -- start 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 -- start start 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc1019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 0x7fbc101a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d380 con 0x7fbc10103290 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d200 con 0x7fbc101028d0 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.040+0000 7fbc0f200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010d500 con 0x7fbc101016e0 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37498/0 (socket says 172.21.15.39:37498) 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 -- 172.21.15.39:0/1700633221 learned_addr learned my addr 172.21.15.39:0/1700633221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0d400640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc1019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0e800640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 0x7fbc101a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 -- 172.21.15.39:0/1700633221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 msgr2=0x7fbc1019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc1019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 -- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 msgr2=0x7fbc101a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 0x7fbc101a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0de00640 1 -- 172.21.15.39:0/1700633221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc101a1830 con 0x7fbc101016e0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.041+0000 7fbc0d400640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc1019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc0de00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc0002c260 tx=0x7fbc0002c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc06a00640 1 -- 172.21.15.39:0/1700633221 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00041070 con 0x7fbc101016e0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc06a00640 1 -- 172.21.15.39:0/1700633221 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00004450 con 0x7fbc101016e0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc06a00640 1 -- 172.21.15.39:0/1700633221 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00034440 con 0x7fbc101016e0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc101a1ac0 con 0x7fbc101016e0 2024-03-16T10:08:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.042+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101a1fc0 con 0x7fbc101016e0 2024-03-16T10:08:20.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.045+0000 7fbc179d6740 1 Processor -- start 2024-03-16T10:08:20.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.045+0000 7fbc06a00640 1 -- 172.21.15.39:0/1700633221 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fbc00044070 con 0x7fbc101016e0 2024-03-16T10:08:20.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.045+0000 7fbc179d6740 1 -- start start 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea32830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 0x563c6ea35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c6e991c50 con 0x563c6ea2dd30 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c6e93a9d0 con 0x563c6ea32ea0 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc179d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c6e98dcf0 con 0x563c6ea2d340 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37500/0 (socket says 172.21.15.39:37500) 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc0e800640 1 -- 172.21.15.39:0/3073865754 learned_addr learned my addr 172.21.15.39:0/3073865754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc0de00640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea32830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.046+0000 7fbc0d400640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 0x563c6ea35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 -- 172.21.15.39:0/3073865754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 msgr2=0x563c6ea35330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 0x563c6ea35330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 -- 172.21.15.39:0/3073865754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 msgr2=0x563c6ea32830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea32830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 -- 172.21.15.39:0/3073865754 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6e954f10 con 0x563c6ea2d340 2024-03-16T10:08:20.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0d400640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 0x563c6ea35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc080118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39764dda263ea0e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbbfb000640 1 -- 172.21.15.39:0/3073865754 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08004d20 con 0x563c6ea2d340 2024-03-16T10:08:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbc06a00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 0x7fbbf40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbbfb000640 1 -- 172.21.15.39:0/3073865754 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc08004050 con 0x563c6ea2d340 2024-03-16T10:08:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.047+0000 7fbbfb000640 1 -- 172.21.15.39:0/3073865754 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08005260 con 0x563c6ea2d340 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc06a00640 1 -- 172.21.15.39:0/1700633221 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fbc000f08e0 con 0x7fbc101016e0 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc0de00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 0x7fbbf40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 -- 172.21.15.39:0/3073865754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 msgr2=0x563c6ea2d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009fd0 tx=0x7fbc080118d0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 -- 172.21.15.39:0/3073865754 shutdown_connections 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea32ea0 0x563c6ea35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea32830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3073865754 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2d340 0x563c6ea2d760 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 -- 172.21.15.39:0/3073865754 >> 172.21.15.39:0/3073865754 conn(0x563c6ea288b0 msgr2=0x563c6ea28cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.048+0000 7fbc179d6740 1 -- 172.21.15.39:0/3073865754 shutdown_connections 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.049+0000 7fbc0de00640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 0x7fbbf40a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b98a0 tx=0x7fbc000f6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.049+0000 7fbc179d6740 1 -- 172.21.15.39:0/3073865754 wait complete. 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.049+0000 7fbc179d6740 1 -- start start 2024-03-16T10:08:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.049+0000 7fbc179d6740 1 Processor -- start 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 0x563c6ea374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 0x563c6ea37a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea3ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea35e50 con 0x563c6ea2d340 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c6e94de20 con 0x563c6ea2dd30 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea35fd0 con 0x563c6ea32ea0 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc179d6740 10 client.0 _ll_register_callbacks cb 0x563c6e94d7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 0x563c6ea374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 0x563c6ea37a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 0x563c6ea374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52210/0 (socket says 172.21.15.39:52210) 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0e800640 1 -- 172.21.15.39:0/4132269864 learned_addr learned my addr 172.21.15.39:0/4132269864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0e800640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea3ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 -- 172.21.15.39:0/4132269864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 msgr2=0x563c6ea3ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea3ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 -- 172.21.15.39:0/4132269864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 msgr2=0x563c6ea374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 0x563c6ea374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0d400640 1 -- 172.21.15.39:0/4132269864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6e6a62d0 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.050+0000 7fbc0e800640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea3ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbc0d400640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 0x563c6ea37a30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc012cf0 tx=0x7fbbfc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927f653a09a25294 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbbef400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0120d0 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc08004050 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fbbfc010430 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbfc012270 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbc179d6740 10 client.25214 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c6e7706b0 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.051+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc08004050 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbbfc01c7f0 con 0x563c6ea2dd30 2024-03-16T10:08:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 10 client.25214 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 10 client.25214 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 20 client.25214 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6ea442d0 0x563c6ea45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 4 client.25214 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6e9ef2b0 con 0x563c6ea442d0 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.052+0000 7fbc0e800640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6ea442d0 0x563c6ea45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.053+0000 7fbc0e800640 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6ea442d0 0x563c6ea45740 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd324c476da82ad server_cookie=a7faad4dd5277a65 in_seq=0 out_seq=0 2024-03-16T10:08:20.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.053+0000 7fbbef400640 10 client.25214 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.053+0000 7fbbef400640 1 -- 172.21.15.39:0/4132269864 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563c6e9ef2b0 con 0x563c6ea442d0 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.053+0000 7fbbef400640 10 client.25214 handle_command_reply: tid=0 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.053+0000 7fbc179d6740 1 client.25214 shutdown 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 msgr2=0x563c6ea37a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 0x563c6ea37a30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc012cf0 tx=0x7fbbfc00f790 comp rx=0 tx=0).stop 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 shutdown_connections 2024-03-16T10:08:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6ea442d0 0x563c6ea45740 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea3ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2dd30 0x563c6ea37a30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 --2- 172.21.15.39:0/4132269864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2d340 0x563c6ea374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 >> 172.21.15.39:0/4132269864 conn(0x563c6ea288b0 msgr2=0x563c6ea27a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbbef400640 0 client.25214 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 shutdown_connections 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 1 -- 172.21.15.39:0/4132269864 wait complete. 2024-03-16T10:08:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.054+0000 7fbc179d6740 20 client.25214 trim_cache size 0 max 16384 2024-03-16T10:08:20.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 Processor -- start 2024-03-16T10:08:20.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 -- start start 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea23900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea43a80 con 0x563c6ea2dd30 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea44150 con 0x563c6ea2d340 2024-03-16T10:08:20.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.075+0000 7fbc179d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea44820 con 0x563c6ea32ea0 2024-03-16T10:08:20.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52214/0 (socket says 172.21.15.39:52214) 2024-03-16T10:08:20.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 -- 172.21.15.39:0/907746317 learned_addr learned my addr 172.21.15.39:0/907746317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:20.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0e800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea35290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:47544/0 (socket says 172.21.15.39:47544) 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0e800640 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea23900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 -- 172.21.15.39:0/907746317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 msgr2=0x563c6ea23900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea23900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 -- 172.21.15.39:0/907746317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 msgr2=0x563c6ea35290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea35290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 -- 172.21.15.39:0/907746317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6ea32170 con 0x563c6ea2dd30 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.076+0000 7fbc0d400640 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009cb0 tx=0x7fbbfc01dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ba91c0fe863c95 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.077+0000 7fbbfa600640 1 -- 172.21.15.39:0/907746317 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0050b0 con 0x563c6ea2dd30 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.077+0000 7fbbfa600640 1 -- 172.21.15.39:0/907746317 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc01c420 con 0x563c6ea2dd30 2024-03-16T10:08:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.077+0000 7fbbfa600640 1 -- 172.21.15.39:0/907746317 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003930 con 0x563c6ea2dd30 2024-03-16T10:08:20.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.077+0000 7fbc179d6740 1 -- 172.21.15.39:0/907746317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 msgr2=0x563c6ea3ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.077+0000 7fbc179d6740 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009cb0 tx=0x7fbbfc01dfb0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 -- 172.21.15.39:0/907746317 shutdown_connections 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea32ea0 0x563c6ea23900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea2dd30 0x563c6ea3ea30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 --2- 172.21.15.39:0/907746317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea35290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 -- 172.21.15.39:0/907746317 >> 172.21.15.39:0/907746317 conn(0x563c6ea288b0 msgr2=0x563c6ea430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 -- 172.21.15.39:0/907746317 shutdown_connections 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 -- 172.21.15.39:0/907746317 wait complete. 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 -- start start 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.078+0000 7fbc179d6740 1 Processor -- start 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea3c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 0x563c6eaa8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea455c0 con 0x563c6ea32ea0 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea44ef0 con 0x563c6ea2d340 2024-03-16T10:08:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c6ea45c90 con 0x563c6ea2dd30 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc179d6740 10 client.0 _ll_register_callbacks cb 0x7fbbfc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37536/0 (socket says 172.21.15.39:37536) 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 -- 172.21.15.39:0/3901825586 learned_addr learned my addr 172.21.15.39:0/3901825586 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea3c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 0x563c6eaa8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 -- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 msgr2=0x563c6ea3c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea3c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 -- 172.21.15.39:0/3901825586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 msgr2=0x563c6eaa8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 0x563c6eaa8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.079+0000 7fbc0d400640 1 -- 172.21.15.39:0/3901825586 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6eaa8d80 con 0x563c6ea2dd30 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 0x563c6eaa8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbc0d400640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc01ccb0 tx=0x7fbbfc01cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86e58d1c75cf3ab server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbbef400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0053a0 con 0x563c6ea2dd30 2024-03-16T10:08:20.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbfc01c420 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fbbfc018070 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbbfc040050 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbc179d6740 10 client.25231 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.080+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c6eaaa080 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc01c420 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbbfc033720 con 0x563c6ea2dd30 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 10 client.25231 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 10 client.25231 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 20 client.25231 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6eaaea40 0x563c6eaad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 4 client.25231 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "status"}) -- 0x563c6e99f190 con 0x563c6eaaea40 2024-03-16T10:08:20.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.081+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6eaaea40 0x563c6eaad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc0e800640 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6eaaea40 0x563c6eaad750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4233217f50503e server_cookie=ab4930baf71a83f1 in_seq=0 out_seq=0 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbbef400640 10 client.25231 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbbef400640 1 -- 172.21.15.39:0/3901825586 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x563c6e99f190 con 0x563c6eaaea40 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbbef400640 10 client.25231 handle_command_reply: tid=0 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 client.25231 shutdown 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 msgr2=0x563c6eaa6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc01ccb0 tx=0x7fbbfc01cd30 comp rx=0 tx=0).stop 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 shutdown_connections 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x563c6eaaea40 0x563c6eaad750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c6ea32ea0 0x563c6eaa8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c6ea2dd30 0x563c6eaa6020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 --2- 172.21.15.39:0/3901825586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c6ea2d340 0x563c6ea3c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbbef400640 0 client.25231 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.082+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 >> 172.21.15.39:0/3901825586 conn(0x563c6ea288b0 msgr2=0x563c6ea23900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 shutdown_connections 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc179d6740 1 -- 172.21.15.39:0/3901825586 wait complete. 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc179d6740 20 client.25231 trim_cache size 0 max 16384 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24569, 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 135.852700107, 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:20.088 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 141.03820417 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 msgr2=0x7fbbf40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 0x7fbbf40a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b98a0 tx=0x7fbc000f6530 comp rx=0 tx=0).stop 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 msgr2=0x7fbc10199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc0002c260 tx=0x7fbc0002c670 comp rx=0 tx=0).stop 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 shutdown_connections 2024-03-16T10:08:20.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fbbf40a2880 0x7fbbf40a4d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc10103290 0x7fbc101a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101028d0 0x7fbc1019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 --2- 172.21.15.39:0/1700633221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc101016e0 0x7fbc10199c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.083+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 >> 172.21.15.39:0/1700633221 conn(0x7fbc100fcc50 msgr2=0x7fbc10073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.084+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 shutdown_connections 2024-03-16T10:08:20.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:20.084+0000 7fbc0f200640 1 -- 172.21.15.39:0/1700633221 wait complete. 2024-03-16T10:08:20.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:20 smithi039 ceph-mon[31510]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:20.102 INFO:tasks.cephadm:Setup complete, yielding 2024-03-16T10:08:20.102 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:08:20.111 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:08:20.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-16T10:08:20.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:20 smithi105 ceph-mon[36559]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:20.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:20 smithi194 ceph-mon[36445]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:20.568 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:21 smithi194 ceph-mon[36445]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:21 smithi194 ceph-mon[36445]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:08:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:21 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:08:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:21 smithi039 ceph-mon[31510]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:21 smithi039 ceph-mon[31510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:08:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:21 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:08:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:21 smithi105 ceph-mon[36559]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-03-16T10:08:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:21 smithi105 ceph-mon[36559]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:08:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:21 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:08:22.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 -- 172.21.15.39:0/738805213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8105730 msgr2=0x7f4cc8107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:22.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/738805213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8105730 0x7f4cc8107b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009990 tx=0x7f4cb4009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 -- 172.21.15.39:0/738805213 shutdown_connections 2024-03-16T10:08:22.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/738805213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8108060 0x7f4cc810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/738805213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8102e00 0x7f4cc81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/738805213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8105730 0x7f4cc8107b20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.858+0000 7f4ccfe00700 1 -- 172.21.15.39:0/738805213 >> 172.21.15.39:0/738805213 conn(0x7f4cc80fc830 msgr2=0x7f4cc80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.859+0000 7f4ccfe00700 1 -- 172.21.15.39:0/738805213 shutdown_connections 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.859+0000 7f4ccfe00700 1 -- 172.21.15.39:0/738805213 wait complete. 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.860+0000 7f4ccfe00700 1 Processor -- start 2024-03-16T10:08:22.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.860+0000 7f4ccfe00700 1 -- start start 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.860+0000 7f4ccfe00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.860+0000 7f4ccfe00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8105730 0x7f4cc819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccfe00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8108060 0x7f4cc81a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccfe00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810df20 con 0x7f4cc8108060 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccfe00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810dda0 con 0x7f4cc8102e00 2024-03-16T10:08:22.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccfe00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810e0a0 con 0x7f4cc8105730 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccca00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8105730 0x7f4cc819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:47580/0 (socket says 172.21.15.39:47580) 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 -- 172.21.15.39:0/3987829744 learned_addr learned my addr 172.21.15.39:0/3987829744 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccde00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8108060 0x7f4cc81a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 -- 172.21.15.39:0/3987829744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8105730 msgr2=0x7f4cc819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8105730 0x7f4cc819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 -- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8108060 msgr2=0x7f4cc81a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8108060 0x7f4cc81a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:22.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 -- 172.21.15.39:0/3987829744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc81a23c0 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.861+0000 7f4ccd400700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e9a0 tx=0x7f4cbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.862+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00cbd0 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.862+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc00cd70 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.862+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0187a0 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.862+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc81a26b0 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.862+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81a2c40 con 0x7f4cc8102e00 2024-03-16T10:08:22.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.863+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc8068f90 con 0x7f4cc8102e00 2024-03-16T10:08:22.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.865+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f4cbc0109a0 con 0x7f4cc8102e00 2024-03-16T10:08:22.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.866+0000 7f4cc6000700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 0x7f4cb00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:22.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.866+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f4cbc0ceff0 con 0x7f4cc8102e00 2024-03-16T10:08:22.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.867+0000 7f4ccca00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 0x7f4cb00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:22.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.867+0000 7f4ccca00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 0x7f4cb00a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b2e0 tx=0x7f4cb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:22.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:22.870+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4cbc09bca0 con 0x7f4cc8102e00 2024-03-16T10:08:23.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.117+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4cc8062710 con 0x7f4cb00a2910 2024-03-16T10:08:23.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.119+0000 7f4cc6000700 1 -- 172.21.15.39:0/3987829744 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4cc8062710 con 0x7f4cb00a2910 2024-03-16T10:08:23.121 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2024-03-16T10:08:23.121 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2024-03-16T10:08:23.121 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 msgr2=0x7f4cb00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 0x7f4cb00a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b2e0 tx=0x7f4cb4005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 msgr2=0x7f4cc819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e9a0 tx=0x7f4cbc00ecb0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 shutdown_connections 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4cb00a2910 0x7f4cb00a4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cc8108060 0x7f4cc81a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cc8102e00 0x7f4cc819a760 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 --2- 172.21.15.39:0/3987829744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4cc8105730 0x7f4cc819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.122+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 >> 172.21.15.39:0/3987829744 conn(0x7f4cc80fc830 msgr2=0x7f4cc8102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.123+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 shutdown_connections 2024-03-16T10:08:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:23.123+0000 7f4ccfe00700 1 -- 172.21.15.39:0/3987829744 wait complete. 2024-03-16T10:08:23.459 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:23 smithi039 ceph-mon[31510]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-16T10:08:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:23 smithi105 ceph-mon[36559]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-16T10:08:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:23 smithi194 ceph-mon[36445]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-16T10:08:23.885 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-16T10:08:24.379 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:24.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:24 smithi105 ceph-mon[36559]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:24.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:24 smithi194 ceph-mon[36445]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:24.712 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:24 smithi039 ceph-mon[31510]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:25.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:25 smithi039 ceph-mon[31510]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-16T10:08:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:25 smithi105 ceph-mon[36559]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-16T10:08:25.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:25 smithi194 ceph-mon[36445]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-16T10:08:26.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.518+0000 7fc30f400700 1 -- 172.21.15.39:0/1553768562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 msgr2=0x7fc30810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:26.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.518+0000 7fc30f400700 1 --2- 172.21.15.39:0/1553768562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc30810a470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc30000b0a0 tx=0x7fc30000b3b0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 -- 172.21.15.39:0/1553768562 shutdown_connections 2024-03-16T10:08:26.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 --2- 172.21.15.39:0/1553768562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc30810a470 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 --2- 172.21.15.39:0/1553768562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308102e20 0x7fc308105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 --2- 172.21.15.39:0/1553768562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308105750 0x7fc308107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 -- 172.21.15.39:0/1553768562 >> 172.21.15.39:0/1553768562 conn(0x7fc3080fc790 msgr2=0x7fc3080febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 -- 172.21.15.39:0/1553768562 shutdown_connections 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.519+0000 7fc30f400700 1 -- 172.21.15.39:0/1553768562 wait complete. 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 Processor -- start 2024-03-16T10:08:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 -- start start 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 0x7fc30819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc3081a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810df50 con 0x7fc308102e20 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810ddd0 con 0x7fc308105750 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.520+0000 7fc30f400700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810e0d0 con 0x7fc308108080 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc30ca00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 0x7fc30819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc30d400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc3081a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:47590/0 (socket says 172.21.15.39:47590) 2024-03-16T10:08:26.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 -- 172.21.15.39:0/800778469 learned_addr learned my addr 172.21.15.39:0/800778469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 -- 172.21.15.39:0/800778469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 msgr2=0x7fc3081a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc3081a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 -- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 msgr2=0x7fc30819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 0x7fc30819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc307e00700 1 -- 172.21.15.39:0/800778469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a2320 con 0x7fc308105750 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc30d400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc3081a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.521+0000 7fc30ca00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 0x7fc30819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.522+0000 7fc307e00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9a0 tx=0x7fc2f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.522+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800ca10 con 0x7fc308105750 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.522+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f800cbb0 con 0x7fc308105750 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.522+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8018660 con 0x7fc308105750 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.523+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a2610 con 0x7fc308105750 2024-03-16T10:08:26.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.523+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a2b00 con 0x7fc308105750 2024-03-16T10:08:26.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.524+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc308068f90 con 0x7fc308105750 2024-03-16T10:08:26.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.526+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc2f801f020 con 0x7fc308105750 2024-03-16T10:08:26.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.528+0000 7fc305600700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 0x7fc2f00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:26.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.528+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fc2f80cef50 con 0x7fc308105750 2024-03-16T10:08:26.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.528+0000 7fc30ca00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 0x7fc2f00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:26.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.529+0000 7fc30ca00700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 0x7fc2f00a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011490 tx=0x7fc2fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:26.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.531+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc2f8053740 con 0x7fc308105750 2024-03-16T10:08:26.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.768+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc308062710 con 0x7fc2f00a2890 2024-03-16T10:08:26.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.784+0000 7fc305600700 1 -- 172.21.15.39:0/800778469 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fc308062710 con 0x7fc2f00a2890 2024-03-16T10:08:26.785 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:08:26.785 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (2m) 33s ago 2m 16.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b fb3123655dec 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi105 running (2m) 79s ago 2m 25.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b c62587e2ba24 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi194 running (2m) 79s ago 2m 16.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a01a0d3ac689 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (2m) 33s ago 2m 15.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b f06753bec5dd 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi105 running (2m) 79s ago 2m 15.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1dd69e54c455 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi194 running (2m) 79s ago 2m 15.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 2b39b6917cfb 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (2m) 33s ago 2m 12.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 61a51e22522b 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi105 running (2m) 79s ago 2m 18.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 4bddcc731235 2024-03-16T10:08:26.786 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi194 running (2m) 79s ago 2m 16.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5c5e101e3d63 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (2m) 33s ago 2m 15.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b e85b2522ff1f 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi105 running (3m) 79s ago 3m 16.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a18fcad48ee7 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi194 running (3m) 79s ago 3m 17.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8e89948e3a38 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (11m) 33s ago 11m 515M - 19.0.0-2162-g3d78f5ba be86dc8cb22b d8f20e15c4f3 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi105 *:8443,8765 running (8m) 79s ago 8m 454M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 09abc74760e2 2024-03-16T10:08:26.789 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi194 *:8443,8765 running (8m) 79s ago 8m 454M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 7eb5976ebe75 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (11m) 33s ago 11m 48.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 40522528bdd2 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi105 running (9m) 79s ago 9m 42.3M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b ef1478987f94 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi194 running (9m) 79s ago 9m 42.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 03457543a46d 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (8m) 33s ago 8m 56.1M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 15d0c9408d9f 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi105 running (8m) 79s ago 8m 59.5M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 80fc50a11e1e 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi194 running (7m) 79s ago 7m 50.2M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 6f281fce8970 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (7m) 33s ago 7m 44.8M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 26de97dd3690 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi105 running (6m) 79s ago 6m 45.4M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 45828de03626 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi194 running (6m) 79s ago 6m 46.8M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b a3db471dc753 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (5m) 33s ago 5m 53.7M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b e6c196c1ee84 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi105 running (5m) 79s ago 5m 44.8M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 6f87be7c79cc 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi194 running (4m) 79s ago 4m 54.0M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae689398ce1b 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (4m) 33s ago 4m 42.3M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b bf04eb387df5 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi105 running (3m) 79s ago 3m 43.4M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 4a06c07ca4f9 2024-03-16T10:08:26.790 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi194 running (3m) 79s ago 3m 43.3M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 27f95af5f0de 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 msgr2=0x7fc2f00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 0x7fc2f00a4d50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011490 tx=0x7fc2fc009260 comp rx=0 tx=0).stop 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 msgr2=0x7fc30819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e9a0 tx=0x7fc2f800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 shutdown_connections 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc2f00a2890 0x7fc2f00a4d50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc308108080 0x7fc3081a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc308102e20 0x7fc30819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 --2- 172.21.15.39:0/800778469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc308105750 0x7fc30819ac00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:26.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 >> 172.21.15.39:0/800778469 conn(0x7fc3080fc790 msgr2=0x7fc308102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:26.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 shutdown_connections 2024-03-16T10:08:26.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:26.787+0000 7fc30f400700 1 -- 172.21.15.39:0/800778469 wait complete. 2024-03-16T10:08:27.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:27 smithi039 ceph-mon[31510]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:27.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:27 smithi039 ceph-mon[31510]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:27.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-16T10:08:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:27 smithi105 ceph-mon[36559]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:27.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:27 smithi105 ceph-mon[36559]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:27 smithi194 ceph-mon[36445]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:27 smithi194 ceph-mon[36445]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:28.018 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:29 smithi105 ceph-mon[36559]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:29.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:29 smithi194 ceph-mon[36445]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:29.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:29 smithi039 ceph-mon[31510]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 -- 172.21.15.39:0/1912766235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 msgr2=0x7f3fe0107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 --2- 172.21.15.39:0/1912766235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe0107ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009990 tx=0x7f3fcc009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 -- 172.21.15.39:0/1912766235 shutdown_connections 2024-03-16T10:08:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 --2- 172.21.15.39:0/1912766235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 0x7f3fe010a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 --2- 172.21.15.39:0/1912766235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 0x7f3fe0105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 --2- 172.21.15.39:0/1912766235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe0107ab0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.312+0000 7f3fe6200700 1 -- 172.21.15.39:0/1912766235 >> 172.21.15.39:0/1912766235 conn(0x7f3fe00fc7a0 msgr2=0x7f3fe00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.313+0000 7f3fe6200700 1 -- 172.21.15.39:0/1912766235 shutdown_connections 2024-03-16T10:08:30.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.314+0000 7f3fe6200700 1 -- 172.21.15.39:0/1912766235 wait complete. 2024-03-16T10:08:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.315+0000 7f3fe6200700 1 Processor -- start 2024-03-16T10:08:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.315+0000 7f3fe6200700 1 -- start start 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.315+0000 7f3fe6200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 0x7f3fe019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fe6200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fe6200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 0x7f3fe01a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fe6200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010df10 con 0x7f3fe0102d90 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fe6200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010dd90 con 0x7f3fe0107ff0 2024-03-16T10:08:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fe6200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010e090 con 0x7f3fe01056c0 2024-03-16T10:08:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fdea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fdf400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 0x7f3fe019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fdea00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:35756/0 (socket says 172.21.15.39:35756) 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fdea00700 1 -- 172.21.15.39:0/2081549668 learned_addr learned my addr 172.21.15.39:0/2081549668 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.316+0000 7f3fdfe00700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 0x7f3fe01a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 -- 172.21.15.39:0/2081549668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 msgr2=0x7f3fe01a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 0x7f3fe01a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 -- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 msgr2=0x7f3fe019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 0x7f3fe019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 -- 172.21.15.39:0/2081549668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a23d0 con 0x7f3fe01056c0 2024-03-16T10:08:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fdea00700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00bc10 tx=0x7f3fcc005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc026070 con 0x7f3fe01056c0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fcc003d70 con 0x7f3fe01056c0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.317+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a2660 con 0x7f3fe01056c0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.318+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a2b10 con 0x7f3fe01056c0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.318+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc021410 con 0x7f3fe01056c0 2024-03-16T10:08:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.319+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe0068f90 con 0x7f3fe01056c0 2024-03-16T10:08:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.321+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3fcc029070 con 0x7f3fe01056c0 2024-03-16T10:08:30.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.322+0000 7f3fd7e00700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 0x7f3fc40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:30.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.322+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f3fcc0cfc70 con 0x7f3fe01056c0 2024-03-16T10:08:30.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.322+0000 7f3fdf400700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 0x7f3fc40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:30.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.323+0000 7f3fdf400700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 0x7f3fc40a4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000f540 tx=0x7f3fd000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:30.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.326+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3fcc05d740 con 0x7f3fe01056c0 2024-03-16T10:08:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.564+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3fe0062710 con 0x7f3fc40a2910 2024-03-16T10:08:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.582+0000 7f3fd7e00700 1 -- 172.21.15.39:0/2081549668 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3fe0062710 con 0x7f3fc40a2910 2024-03-16T10:08:30.582 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-16T10:08:30.582 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 83s ago 3m smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi105=b;smithi105=e;smithi105=h;smithi105=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T10:08:30.582 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 83s ago 9m smithi039=x;smithi105=y;smithi194=z;count:3 2024-03-16T10:08:30.583 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 83s ago 9m smithi039:172.21.15.39=a;smithi105:172.21.15.105=b;smithi194:172.21.15.194=c;count:3 2024-03-16T10:08:30.583 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 83s ago - 2024-03-16T10:08:30.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 msgr2=0x7f3fc40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:30.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 0x7f3fc40a4dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000f540 tx=0x7f3fd000b040 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 msgr2=0x7f3fe019acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00bc10 tx=0x7f3fcc005f10 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 shutdown_connections 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3fc40a2910 0x7f3fc40a4dd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fe0107ff0 0x7f3fe01a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe0102d90 0x7f3fe019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 --2- 172.21.15.39:0/2081549668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fe01056c0 0x7f3fe019acb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 >> 172.21.15.39:0/2081549668 conn(0x7f3fe00fc7a0 msgr2=0x7f3fe01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 shutdown_connections 2024-03-16T10:08:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:30.585+0000 7f3fe6200700 1 -- 172.21.15.39:0/2081549668 wait complete. 2024-03-16T10:08:31.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-16T10:08:31.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:31 smithi105 ceph-mon[36559]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:31.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:31 smithi105 ceph-mon[36559]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:31.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:31 smithi194 ceph-mon[36445]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:31.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:31 smithi194 ceph-mon[36445]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:31.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:31 smithi039 ceph-mon[31510]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:31.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:31 smithi039 ceph-mon[31510]: from='client.25249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:33.625 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:33 smithi039 ceph-mon[31510]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:33.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:33 smithi194 ceph-mon[36445]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:33 smithi105 ceph-mon[36559]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:33.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 -- 172.21.15.39:0/434503448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4101540 msgr2=0x7f71b41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 --2- 172.21.15.39:0/434503448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4101540 0x7f71b41019c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009990 tx=0x7f71a0009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 -- 172.21.15.39:0/434503448 shutdown_connections 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 --2- 172.21.15.39:0/434503448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 0x7f71b4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 --2- 172.21.15.39:0/434503448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4100be0 0x7f71b4101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 --2- 172.21.15.39:0/434503448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4101540 0x7f71b41019c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.923+0000 7f71bb400700 1 -- 172.21.15.39:0/434503448 >> 172.21.15.39:0/434503448 conn(0x7f71b40fc7a0 msgr2=0x7f71b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:33.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.924+0000 7f71bb400700 1 -- 172.21.15.39:0/434503448 shutdown_connections 2024-03-16T10:08:33.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.924+0000 7f71bb400700 1 -- 172.21.15.39:0/434503448 wait complete. 2024-03-16T10:08:33.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.925+0000 7f71bb400700 1 Processor -- start 2024-03-16T10:08:33.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.925+0000 7f71bb400700 1 -- start start 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.925+0000 7f71bb400700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.925+0000 7f71bb400700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 0x7f71b419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.925+0000 7f71bb400700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 0x7f71b41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36104/0 (socket says 172.21.15.39:36104) 2024-03-16T10:08:33.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 0x7f71b419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 -- 172.21.15.39:0/2287687589 learned_addr learned my addr 172.21.15.39:0/2287687589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b3e00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 0x7f71b419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:35766/0 (socket says 172.21.15.39:35766) 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4110100 con 0x7f71b4101f00 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410ff80 con 0x7f71b4100be0 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4110280 con 0x7f71b4101540 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b9400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 0x7f71b41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:33.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 -- 172.21.15.39:0/2287687589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 msgr2=0x7f71b419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 0x7f71b419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 -- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 msgr2=0x7f71b41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 0x7f71b41a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.926+0000 7f71b8a00700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b41a23e0 con 0x7f71b4100be0 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71b8a00700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e9a0 tx=0x7f71a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a800cbd0 con 0x7f71b4100be0 2024-03-16T10:08:33.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b41a26d0 con 0x7f71b4100be0 2024-03-16T10:08:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a800cd70 con 0x7f71b4100be0 2024-03-16T10:08:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a80187a0 con 0x7f71b4100be0 2024-03-16T10:08:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.927+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a2c10 con 0x7f71b4100be0 2024-03-16T10:08:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.928+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b4068f90 con 0x7f71b4100be0 2024-03-16T10:08:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.930+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f71a80109a0 con 0x7f71b4100be0 2024-03-16T10:08:33.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.931+0000 7f71b1600700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 0x7f719c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:33.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.932+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f71a80cf080 con 0x7f71b4100be0 2024-03-16T10:08:33.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.932+0000 7f71b3e00700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 0x7f719c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:33.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.932+0000 7f71b3e00700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 0x7f719c0a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b7b0 tx=0x7f71a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:33.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:33.940+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f71a809bd10 con 0x7f71b4100be0 2024-03-16T10:08:34.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.177+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f71b4062710 con 0x7f719c0a2890 2024-03-16T10:08:34.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.180+0000 7f71b1600700 1 -- 172.21.15.39:0/2287687589 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f71b4062710 con 0x7f719c0a2890 2024-03-16T10:08:34.181 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2024-03-16T10:08:34.181 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2024-03-16T10:08:34.181 INFO:teuthology.orchestra.run.smithi039.stdout:smithi105 172.21.15.105 2024-03-16T10:08:34.181 INFO:teuthology.orchestra.run.smithi039.stdout:smithi194 172.21.15.194 2024-03-16T10:08:34.181 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 msgr2=0x7f719c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 0x7f719c0a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b7b0 tx=0x7f71a0005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 msgr2=0x7f71b419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e9a0 tx=0x7f71a800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 shutdown_connections 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f719c0a2890 0x7f719c0a4d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71b4101f00 0x7f71b41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b4100be0 0x7f71b419a780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 --2- 172.21.15.39:0/2287687589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71b4101540 0x7f71b419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 >> 172.21.15.39:0/2287687589 conn(0x7f71b40fc7a0 msgr2=0x7f71b40fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.183+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 shutdown_connections 2024-03-16T10:08:34.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:34.184+0000 7f71bb400700 1 -- 172.21.15.39:0/2287687589 wait complete. 2024-03-16T10:08:34.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-16T10:08:35.465 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:35.633 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:35 smithi039 ceph-mon[31510]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:35.633 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:35 smithi039 ceph-mon[31510]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:35 smithi194 ceph-mon[36445]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:35.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:35 smithi194 ceph-mon[36445]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:35 smithi105 ceph-mon[36559]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:35 smithi105 ceph-mon[36559]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:37.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:37 smithi039 ceph-mon[31510]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:37.666 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:37 smithi105 ceph-mon[36559]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 -- 172.21.15.39:0/310976905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0102cb0 msgr2=0x7ff3d01050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 --2- 172.21.15.39:0/310976905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0102cb0 0x7ff3d01050a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009990 tx=0x7ff3c0009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 -- 172.21.15.39:0/310976905 shutdown_connections 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 --2- 172.21.15.39:0/310976905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0107f10 0x7ff3d010a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 --2- 172.21.15.39:0/310976905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0102cb0 0x7ff3d01050a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 --2- 172.21.15.39:0/310976905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d01079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.684+0000 7ff3cf200700 1 -- 172.21.15.39:0/310976905 >> 172.21.15.39:0/310976905 conn(0x7ff3d00fc6c0 msgr2=0x7ff3d00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.685+0000 7ff3cf200700 1 -- 172.21.15.39:0/310976905 shutdown_connections 2024-03-16T10:08:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.685+0000 7ff3cf200700 1 -- 172.21.15.39:0/310976905 wait complete. 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.686+0000 7ff3cf200700 1 Processor -- start 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.686+0000 7ff3cf200700 1 -- start start 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0102cb0 0x7ff3d0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010de80 con 0x7ff3d0102cb0 2024-03-16T10:08:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010dd00 con 0x7ff3d0107f10 2024-03-16T10:08:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cf200700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d010e000 con 0x7ff3d01055e0 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3cde00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0102cb0 0x7ff3d0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.687+0000 7ff3ce800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36116/0 (socket says 172.21.15.39:36116) 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 -- 172.21.15.39:0/2547088901 learned_addr learned my addr 172.21.15.39:0/2547088901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3cd400700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 -- 172.21.15.39:0/2547088901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 msgr2=0x7ff3d0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 -- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0102cb0 msgr2=0x7ff3d0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0102cb0 0x7ff3d0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 -- 172.21.15.39:0/2547088901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d01a0060 con 0x7ff3d0107f10 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3cd400700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.688+0000 7ff3ce800700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800ef90 tx=0x7ff3c800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.689+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80092e0 con 0x7ff3d0107f10 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.689+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8004bb0 con 0x7ff3d0107f10 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.689+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8004280 con 0x7ff3d0107f10 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.689+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01a02f0 con 0x7ff3d0107f10 2024-03-16T10:08:37.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.689+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a07e0 con 0x7ff3d0107f10 2024-03-16T10:08:37.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.690+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d0068f90 con 0x7ff3d0107f10 2024-03-16T10:08:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.692+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff3c801b030 con 0x7ff3d0107f10 2024-03-16T10:08:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.693+0000 7ff3c6a00700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 0x7ff3b40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.694+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7ff3c80ceab0 con 0x7ff3d0107f10 2024-03-16T10:08:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.694+0000 7ff3cde00700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 0x7ff3b40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.695+0000 7ff3cde00700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 0x7ff3b40a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006010 tx=0x7ff3c001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:37.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.697+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff3c8022020 con 0x7ff3d0107f10 2024-03-16T10:08:37.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:37 smithi194 ceph-mon[36445]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.958+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff3d0062710 con 0x7ff3b40a2910 2024-03-16T10:08:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.962+0000 7ff3c6a00700 1 -- 172.21.15.39:0/2547088901 <== mgr.14152 v2:172.21.15.39:6800/1082942412 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff3d0062710 con 0x7ff3b40a2910 2024-03-16T10:08:37.963 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-16T10:08:37.963 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 4m ago Has a FileSystem, LVM detected 2024-03-16T10:08:37.963 INFO:teuthology.orchestra.run.smithi039.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-16T10:08:37.963 INFO:teuthology.orchestra.run.smithi039.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 msgr2=0x7ff3b40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 0x7ff3b40a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006010 tx=0x7ff3c001a040 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 msgr2=0x7ff3d019f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800ef90 tx=0x7ff3c800c450 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 shutdown_connections 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff3b40a2910 0x7ff3b40a4dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3d0107f10 0x7ff3d019f9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d0102cb0 0x7ff3d0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 --2- 172.21.15.39:0/2547088901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3d01055e0 0x7ff3d0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 >> 172.21.15.39:0/2547088901 conn(0x7ff3d00fc6c0 msgr2=0x7ff3d0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.965+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 shutdown_connections 2024-03-16T10:08:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:37.966+0000 7ff3cf200700 1 -- 172.21.15.39:0/2547088901 wait complete. 2024-03-16T10:08:38.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:38 smithi039 ceph-mon[31510]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:38.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:38 smithi194 ceph-mon[36445]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:38.779 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:08:38.787 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:08:38.787 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T10:08:38.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:38 smithi105 ceph-mon[36559]: from='client.25250 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:08:39.254 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:39 smithi194 ceph-mon[36445]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:39.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:39 smithi039 ceph-mon[31510]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:39 smithi105 ceph-mon[36559]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.670+0000 7f7f07c00700 1 -- 172.21.15.39:0/232597896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 msgr2=0x7f7f000751b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.670+0000 7f7f07c00700 1 --2- 172.21.15.39:0/232597896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f000751b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009990 tx=0x7f7eec009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 -- 172.21.15.39:0/232597896 shutdown_connections 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 --2- 172.21.15.39:0/232597896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 0x7f7f00110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 --2- 172.21.15.39:0/232597896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 0x7f7f00074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 --2- 172.21.15.39:0/232597896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f000751b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 -- 172.21.15.39:0/232597896 >> 172.21.15.39:0/232597896 conn(0x7f7f000fd870 msgr2=0x7f7f000ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:41.673 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:41 smithi039 ceph-mon[31510]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:41.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 -- 172.21.15.39:0/232597896 shutdown_connections 2024-03-16T10:08:41.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.671+0000 7f7f07c00700 1 -- 172.21.15.39:0/232597896 wait complete. 2024-03-16T10:08:41.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.672+0000 7f7f07c00700 1 Processor -- start 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 -- start start 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 0x7f7f0019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 0x7f7f001a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00114560 con 0x7f7f00076300 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f001143e0 con 0x7f7f00074d30 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f07c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f001146e0 con 0x7f7f0010e8f0 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f05200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:41.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f05200700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44040/0 (socket says 172.21.15.39:44040) 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f05200700 1 -- 172.21.15.39:0/3517054638 learned_addr learned my addr 172.21.15.39:0/3517054638 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.673+0000 7f7f05c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 0x7f7f001a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7effe00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 0x7f7f0019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 -- 172.21.15.39:0/3517054638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 msgr2=0x7f7f001a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 0x7f7f001a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 -- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 msgr2=0x7f7f0019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 0x7f7f0019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 -- 172.21.15.39:0/3517054638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f001a6800 con 0x7f7f00074d30 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7f05200700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e970 tx=0x7f7ef400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.674+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004a30 con 0x7f7f00074d30 2024-03-16T10:08:41.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.675+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef400ccc0 con 0x7f7f00074d30 2024-03-16T10:08:41.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.675+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f001a6af0 con 0x7f7f00074d30 2024-03-16T10:08:41.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.675+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001a7080 con 0x7f7f00074d30 2024-03-16T10:08:41.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.675+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40055f0 con 0x7f7f00074d30 2024-03-16T10:08:41.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.676+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee0005360 con 0x7f7f00074d30 2024-03-16T10:08:41.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.678+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7ef400c7e0 con 0x7f7f00074d30 2024-03-16T10:08:41.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.679+0000 7f7efd600700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 0x7f7ee80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:41.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.679+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f7ef40cf0f0 con 0x7f7f00074d30 2024-03-16T10:08:41.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.679+0000 7f7effe00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 0x7f7ee80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:41.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.680+0000 7f7effe00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 0x7f7ee80a4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00b7b0 tx=0x7f7eec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:41.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.683+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7ef409bd80 con 0x7f7f00074d30 2024-03-16T10:08:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:41 smithi194 ceph-mon[36445]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:41 smithi105 ceph-mon[36559]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:41.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.970+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7ee0005e60 con 0x7f7f00074d30 2024-03-16T10:08:41.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.971+0000 7f7efd600700 1 -- 172.21.15.39:0/3517054638 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3098 (secure 0 0 0) 0x7f7ef409b680 con 0x7f7f00074d30 2024-03-16T10:08:41.972 INFO:teuthology.orchestra.run.smithi039.stdout:e28 2024-03-16T10:08:41.972 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:08:41.972 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 28 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-03-16T10:06:03.734324+0000 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-03-16T10:06:40.587364+0000 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2024-03-16T10:08:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24569,1=24557,2=24544,3=24523,4=24518} 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3,4] 2024-03-16T10:08:41.974 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:qdb_cluster leader: 24569 members: 24557,24569,24518,24523,24544 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{0:24511} state up:standby-replay seq 1 addr [v2:172.21.15.194:6834/3003167044,v1:172.21.15.194:6835/3003167044] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{1:24557} state up:active seq 12 addr [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{1:24485} state up:standby-replay seq 1 addr [v2:172.21.15.105:6834/106855660,v1:172.21.15.105:6835/106855660] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{2:24544} state up:active seq 14 addr [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{2:24467} state up:standby-replay seq 1 addr [v2:172.21.15.39:6836/2605661802,v1:172.21.15.39:6837/2605661802] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{3:24523} state up:active seq 16 addr [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{3:24455} state up:standby-replay seq 1 addr [v2:172.21.15.39:6834/1951962450,v1:172.21.15.39:6835/1951962450] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.975 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{4:24518} state up:active seq 15 addr [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{4:24445} state up:standby-replay seq 1 addr [v2:172.21.15.194:6832/1279561047,v1:172.21.15.194:6833/1279561047] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{-1:14457} state up:standby seq 1 addr [v2:172.21.15.39:6838/428058525,v1:172.21.15.39:6839/428058525] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.105:6832/2269577878,v1:172.21.15.105:6833/2269577878] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:08:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.974+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 msgr2=0x7f7ee80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.974+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 0x7f7ee80a4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00b7b0 tx=0x7f7eec005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 msgr2=0x7f7f0019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e970 tx=0x7f7ef400ec80 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 shutdown_connections 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f7ee80a2910 0x7f7ee80a4dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f0010e8f0 0x7f7f001a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f00074d30 0x7f7f0019eba0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 --2- 172.21.15.39:0/3517054638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f00076300 0x7f7f0019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 >> 172.21.15.39:0/3517054638 conn(0x7f7f000fd870 msgr2=0x7f7f00101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 shutdown_connections 2024-03-16T10:08:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:41.975+0000 7f7f07c00700 1 -- 172.21.15.39:0/3517054638 wait complete. 2024-03-16T10:08:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:42.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:42 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3517054638' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:08:42.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:42 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3517054638' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:08:42.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-16T10:08:42.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:42 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3517054638' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:08:43.227 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:43.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:43 smithi194 ceph-mon[36445]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:43 smithi039 ceph-mon[31510]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:43 smithi105 ceph-mon[36559]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:45.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 -- 172.21.15.39:0/3911528341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 msgr2=0x7efdcc107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:45.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 --2- 172.21.15.39:0/3911528341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc107ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009990 tx=0x7efdb8009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 -- 172.21.15.39:0/3911528341 shutdown_connections 2024-03-16T10:08:45.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 --2- 172.21.15.39:0/3911528341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc107ff0 0x7efdcc10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 --2- 172.21.15.39:0/3911528341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc102d90 0x7efdcc105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 --2- 172.21.15.39:0/3911528341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc107ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 -- 172.21.15.39:0/3911528341 >> 172.21.15.39:0/3911528341 conn(0x7efdcc0fc7a0 msgr2=0x7efdcc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 -- 172.21.15.39:0/3911528341 shutdown_connections 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.657+0000 7efdd1c00700 1 -- 172.21.15.39:0/3911528341 wait complete. 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.658+0000 7efdd1c00700 1 Processor -- start 2024-03-16T10:08:45.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 -- start start 2024-03-16T10:08:45.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc102d90 0x7efdcc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:45.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 0x7efdcc1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10df60 con 0x7efdcc102d90 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10dde0 con 0x7efdcc1056c0 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.659+0000 7efdd1c00700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc10e0e0 con 0x7efdcc107ff0 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44046/0 (socket says 172.21.15.39:44046) 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 -- 172.21.15.39:0/2811152589 learned_addr learned my addr 172.21.15.39:0/2811152589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdcaa00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc102d90 0x7efdcc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdcb400700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 0x7efdcc1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 -- 172.21.15.39:0/2811152589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 msgr2=0x7efdcc1a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 0x7efdcc1a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 -- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc102d90 msgr2=0x7efdcc19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc102d90 0x7efdcc19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 -- 172.21.15.39:0/2811152589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc1a2450 con 0x7efdcc1056c0 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdcb400700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 0x7efdcc1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.660+0000 7efdca000700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005910 tx=0x7efdb800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.661+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8026070 con 0x7efdcc1056c0 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.661+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdb800b610 con 0x7efdcc1056c0 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.661+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8021670 con 0x7efdcc1056c0 2024-03-16T10:08:45.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.662+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc1a26e0 con 0x7efdcc1056c0 2024-03-16T10:08:45.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.662+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc1a2ae0 con 0x7efdcc1056c0 2024-03-16T10:08:45.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.663+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdcc068f90 con 0x7efdcc1056c0 2024-03-16T10:08:45.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.666+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7efdb8009d10 con 0x7efdcc1056c0 2024-03-16T10:08:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.668+0000 7efdc3400700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 0x7efdb00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.668+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7efdb80cf720 con 0x7efdcc1056c0 2024-03-16T10:08:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.668+0000 7efdcaa00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 0x7efdb00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:45.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.669+0000 7efdcaa00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 0x7efdb00a4dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009bf0 tx=0x7efdbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.676+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efdb8013070 con 0x7efdcc1056c0 2024-03-16T10:08:45.677 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:45 smithi039 ceph-mon[31510]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:45.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:45 smithi194 ceph-mon[36445]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:45 smithi105 ceph-mon[36559]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:45.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.947+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7efdcc062ee0 con 0x7efdcc1056c0 2024-03-16T10:08:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.948+0000 7efdc3400700 1 -- 172.21.15.39:0/2811152589 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5707 (secure 0 0 0) 0x7efdb809c440 con 0x7efdcc1056c0 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 94 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-03-16T09:57:03.274650+0000 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-03-16T10:08:14.004515+0000 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 27 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-03-16T10:08:45.949 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/1765190956,v1:172.21.15.39:6803/1765190956] [v2:172.21.15.39:6804/1765190956,v1:172.21.15.39:6805/1765190956] exists,up f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/3759059886,v1:172.21.15.105:6801/3759059886] [v2:172.21.15.105:6802/3759059886,v1:172.21.15.105:6803/3759059886] exists,up 097d0219-dbb6-485b-a385-33b389bf02c7 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/3029390216,v1:172.21.15.194:6801/3029390216] [v2:172.21.15.194:6802/3029390216,v1:172.21.15.194:6803/3029390216] exists,up 1f3873f3-0b87-42c9-a34b-f62099b77f39 2024-03-16T10:08:45.950 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 26 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/2181780282,v1:172.21.15.39:6811/2181780282] [v2:172.21.15.39:6812/2181780282,v1:172.21.15.39:6813/2181780282] exists,up a6455fd4-f277-4db1-a5c8-0fb262d3bb49 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 34 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/45639064,v1:172.21.15.105:6809/45639064] [v2:172.21.15.105:6810/45639064,v1:172.21.15.105:6811/45639064] exists,up 3d166e33-4a4b-48ce-b293-7d6fe30b2adb 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/846499342,v1:172.21.15.194:6809/846499342] [v2:172.21.15.194:6810/846499342,v1:172.21.15.194:6811/846499342] exists,up 2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/2687294555,v1:172.21.15.39:6819/2687294555] [v2:172.21.15.39:6820/2687294555,v1:172.21.15.39:6821/2687294555] exists,up f4d56835-6eb8-4763-bc1e-da513d084bb7 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/2696110582,v1:172.21.15.105:6817/2696110582] [v2:172.21.15.105:6818/2696110582,v1:172.21.15.105:6819/2696110582] exists,up 14b5763b-c967-49ad-a8d6-e37f49999f9c 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/4066786561,v1:172.21.15.194:6817/4066786561] [v2:172.21.15.194:6818/4066786561,v1:172.21.15.194:6819/4066786561] exists,up 9e0fa1af-48e7-4764-ad45-390850e2bc7a 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/2744339033,v1:172.21.15.39:6827/2744339033] [v2:172.21.15.39:6828/2744339033,v1:172.21.15.39:6829/2744339033] exists,up c112bf93-06c2-409e-8f31-837fd3d2b600 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/4246880500,v1:172.21.15.105:6825/4246880500] [v2:172.21.15.105:6826/4246880500,v1:172.21.15.105:6827/4246880500] exists,up f44ef9e2-7391-4298-a185-199839c37867 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/254372384,v1:172.21.15.194:6825/254372384] [v2:172.21.15.194:6826/254372384,v1:172.21.15.194:6827/254372384] exists,up 9a171911-8bf9-4ad4-ab72-98d5c48d0807 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.4 [2,11] 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.b [2,4] 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.10 [8,11] 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.0 [5,9] 2024-03-16T10:08:45.951 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.e [4,6] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.17 [4,7] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.1d [4,6,5,9] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.23 [4,8] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.2a [3,8] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.3b [3,9] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.10 [9,8] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.11 [5,8] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.18 [9,10] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.19 [9,8] 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3114779354 expires 2024-03-17T09:58:12.056328+0000 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3549110681 expires 2024-03-17T09:58:12.056328+0000 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1618736720 expires 2024-03-17T09:58:12.056328+0000 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/1974903299 expires 2024-03-17T09:58:12.056328+0000 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/865907722 expires 2024-03-17T09:57:37.234906+0000 2024-03-16T10:08:45.952 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/4208944392 expires 2024-03-17T09:57:37.234906+0000 2024-03-16T10:08:45.953 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/3969870217 expires 2024-03-17T09:57:37.234906+0000 2024-03-16T10:08:45.953 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/3969870217 expires 2024-03-17T09:57:37.234906+0000 2024-03-16T10:08:45.953 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3617878948 expires 2024-03-17T09:57:37.234906+0000 2024-03-16T10:08:45.953 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/1974903299 expires 2024-03-17T09:58:12.056328+0000 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 msgr2=0x7efdb00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 0x7efdb00a4dd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdbc009bf0 tx=0x7efdbc009260 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 msgr2=0x7efdcc19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005910 tx=0x7efdb800bc60 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 shutdown_connections 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7efdb00a2910 0x7efdb00a4dd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdcc107ff0 0x7efdcc1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdcc102d90 0x7efdcc19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 --2- 172.21.15.39:0/2811152589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efdcc1056c0 0x7efdcc19ad30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 >> 172.21.15.39:0/2811152589 conn(0x7efdcc0fc7a0 msgr2=0x7efdcc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 shutdown_connections 2024-03-16T10:08:45.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:45.951+0000 7efdd1c00700 1 -- 172.21.15.39:0/2811152589 wait complete. 2024-03-16T10:08:46.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:46 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2811152589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T10:08:46.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:46 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2811152589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T10:08:46.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:46 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2811152589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T10:08:46.917 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-16T10:08:46.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 -- 172.21.15.39:0/1698410425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c1007c0 msgr2=0x7fc40c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 --2- 172.21.15.39:0/1698410425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c1007c0 0x7fc40c100c20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009860 tx=0x7fc3f802bb70 comp rx=0 tx=0).stop 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 -- 172.21.15.39:0/1698410425 shutdown_connections 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 --2- 172.21.15.39:0/1698410425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 0x7fc40c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 --2- 172.21.15.39:0/1698410425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c1007c0 0x7fc40c100c20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 --2- 172.21.15.39:0/1698410425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c0ffe60 0x7fc40c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.140+0000 7fc411000640 1 -- 172.21.15.39:0/1698410425 >> 172.21.15.39:0/1698410425 conn(0x7fc40c0fba60 msgr2=0x7fc40c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:47.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.141+0000 7fc411000640 1 -- 172.21.15.39:0/1698410425 shutdown_connections 2024-03-16T10:08:47.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.141+0000 7fc411000640 1 -- 172.21.15.39:0/1698410425 wait complete. 2024-03-16T10:08:47.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.142+0000 7fc411000640 1 Processor -- start 2024-03-16T10:08:47.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.142+0000 7fc411000640 1 -- start start 2024-03-16T10:08:47.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.142+0000 7fc411000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.142+0000 7fc411000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c1007c0 0x7fc40c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.142+0000 7fc411000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 0x7fc40c1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc411000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c10f430 con 0x7fc40c101160 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc411000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c10f2b0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc411000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c10f5b0 con 0x7fc40c1007c0 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44072/0 (socket says 172.21.15.39:44072) 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 -- 172.21.15.39:0/2436776230 learned_addr learned my addr 172.21.15.39:0/2436776230 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40be00640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 0x7fc40c1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40aa00640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c1007c0 0x7fc40c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 -- 172.21.15.39:0/2436776230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c1007c0 msgr2=0x7fc40c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c1007c0 0x7fc40c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 -- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 msgr2=0x7fc40c1a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 0x7fc40c1a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 -- 172.21.15.39:0/2436776230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c1a1670 con 0x7fc40c0ffe60 2024-03-16T10:08:47.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.143+0000 7fc40b400640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9c0 tx=0x7fc3fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:47.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.144+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00cde0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.144+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3fc0035f0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.144+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc010690 con 0x7fc40c0ffe60 2024-03-16T10:08:47.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.144+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c1a1960 con 0x7fc40c0ffe60 2024-03-16T10:08:47.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.144+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c1a1e80 con 0x7fc40c0ffe60 2024-03-16T10:08:47.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.145+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c0684f0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.147+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fc3fc0037a0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.148+0000 7fc403e00640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 0x7fc3f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.148+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fc3fc0cf2b0 con 0x7fc40c0ffe60 2024-03-16T10:08:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.149+0000 7fc40aa00640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 0x7fc3f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.150+0000 7fc40aa00640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 0x7fc3f00a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009830 tx=0x7fc3f8034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:47.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.156+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc3fc09bf40 con 0x7fc40c0ffe60 2024-03-16T10:08:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.360+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc40c062480 con 0x7fc40c0ffe60 2024-03-16T10:08:47.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.362+0000 7fc403e00640 1 -- 172.21.15.39:0/2436776230 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24850 (secure 0 0 0) 0x7fc3fc09b840 con 0x7fc40c0ffe60 2024-03-16T10:08:47.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:47.364 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:08:14.004515+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 msgr2=0x7fc3f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 0x7fc3f00a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009830 tx=0x7fc3f8034ad0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 msgr2=0x7fc40c199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00e9c0 tx=0x7fc3fc00ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 shutdown_connections 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc3f00a2800 0x7fc3f00a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc40c101160 0x7fc40c1a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc40c1007c0 0x7fc40c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 --2- 172.21.15.39:0/2436776230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc40c0ffe60 0x7fc40c199a50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 >> 172.21.15.39:0/2436776230 conn(0x7fc40c0fba60 msgr2=0x7fc40c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 shutdown_connections 2024-03-16T10:08:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.365+0000 7fc411000640 1 -- 172.21.15.39:0/2436776230 wait complete. 2024-03-16T10:08:47.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:08:47.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 -- 172.21.15.39:0/2560407497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4104ad0 msgr2=0x7ff0f4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/2560407497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4104ad0 0x7ff0f4104f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009900 tx=0x7ff0dc02bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 -- 172.21.15.39:0/2560407497 shutdown_connections 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/2560407497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f4105490 0x7ff0f410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/2560407497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4104ad0 0x7ff0f4104f50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/2560407497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f41038e0 0x7ff0f4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 -- 172.21.15.39:0/2560407497 >> 172.21.15.39:0/2560407497 conn(0x7ff0f40fee50 msgr2=0x7ff0f4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 -- 172.21.15.39:0/2560407497 shutdown_connections 2024-03-16T10:08:47.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.620+0000 7ff0f3e00640 1 -- 172.21.15.39:0/2560407497 wait complete. 2024-03-16T10:08:47.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.621+0000 7ff0f3e00640 1 Processor -- start 2024-03-16T10:08:47.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.621+0000 7ff0f3e00640 1 -- start start 2024-03-16T10:08:47.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 0x7ff0f4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 0x7ff0f419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 0x7ff0f41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410f5b0 con 0x7ff0f41038e0 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410f430 con 0x7ff0f4104ad0 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410f730 con 0x7ff0f4105490 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 0x7ff0f4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 0x7ff0f4199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55056/0 (socket says 172.21.15.39:55056) 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f2a00640 1 -- 172.21.15.39:0/1792264930 learned_addr learned my addr 172.21.15.39:0/1792264930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 0x7ff0f419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3400640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 0x7ff0f41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 -- 172.21.15.39:0/1792264930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 msgr2=0x7ff0f41a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 0x7ff0f41a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 -- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 msgr2=0x7ff0f4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 0x7ff0f4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0ea000640 1 -- 172.21.15.39:0/1792264930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41a1810 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.622+0000 7ff0f3400640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 0x7ff0f41a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0ea000640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 0x7ff0f419a130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc036040 tx=0x7ff0dc02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc043020 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0dc0054e0 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a1aa0 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc003ca0 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.623+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a1fc0 con 0x7ff0f4104ad0 2024-03-16T10:08:47.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.625+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f410ac00 con 0x7ff0f4104ad0 2024-03-16T10:08:47.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.626+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff0dc046020 con 0x7ff0f4104ad0 2024-03-16T10:08:47.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.627+0000 7ff0f8a00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 0x7ff0d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:47.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.627+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7ff0dc0f0f70 con 0x7ff0f4104ad0 2024-03-16T10:08:47.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.627+0000 7ff0f2a00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 0x7ff0d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:47.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.628+0000 7ff0f2a00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 0x7ff0d80a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005f40 tx=0x7ff0e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.636+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff0dc04d020 con 0x7ff0f4104ad0 2024-03-16T10:08:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:47 smithi194 ceph-mon[36445]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:47 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2436776230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:47 smithi039 ceph-mon[31510]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:47 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2436776230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:47.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.842+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0f40623e0 con 0x7ff0f4104ad0 2024-03-16T10:08:47.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.843+0000 7ff0f8a00640 1 -- 172.21.15.39:0/1792264930 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7ff0dc0bdc00 con 0x7ff0f4104ad0 2024-03-16T10:08:47.843 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:08:47.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 msgr2=0x7ff0d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 0x7ff0d80a4cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005f40 tx=0x7ff0e4009500 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 msgr2=0x7ff0f419a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 0x7ff0f419a130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc036040 tx=0x7ff0dc02c6c0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 shutdown_connections 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7ff0d80a2800 0x7ff0d80a4cc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0f4105490 0x7ff0f41a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4104ad0 0x7ff0f419a130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.845+0000 7ff0f3e00640 1 --2- 172.21.15.39:0/1792264930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0f41038e0 0x7ff0f4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.846+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 >> 172.21.15.39:0/1792264930 conn(0x7ff0f40fee50 msgr2=0x7ff0f4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.846+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 shutdown_connections 2024-03-16T10:08:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:47.846+0000 7ff0f3e00640 1 -- 172.21.15.39:0/1792264930 wait complete. 2024-03-16T10:08:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:47 smithi105 ceph-mon[36559]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:47 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2436776230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:47.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 -- 172.21.15.39:0/1764119455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0105490 msgr2=0x7f13c010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 --2- 172.21.15.39:0/1764119455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0105490 0x7f13c010bd20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009ca0 tx=0x7f13b802c100 comp rx=0 tx=0).stop 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 -- 172.21.15.39:0/1764119455 shutdown_connections 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 --2- 172.21.15.39:0/1764119455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0105490 0x7f13c010bd20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 --2- 172.21.15.39:0/1764119455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0104ad0 0x7f13c0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 --2- 172.21.15.39:0/1764119455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 -- 172.21.15.39:0/1764119455 >> 172.21.15.39:0/1764119455 conn(0x7f13c00fee50 msgr2=0x7f13c0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.090+0000 7f13bf200640 1 -- 172.21.15.39:0/1764119455 shutdown_connections 2024-03-16T10:08:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.091+0000 7f13bf200640 1 -- 172.21.15.39:0/1764119455 wait complete. 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.091+0000 7f13bf200640 1 Processor -- start 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 -- start start 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 0x7f13c01a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13c010f540 con 0x7f13c0105490 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13c010f3c0 con 0x7f13c0104ad0 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bf200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f13c010f6c0 con 0x7f13c01038e0 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bd400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44106/0 (socket says 172.21.15.39:44106) 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bde00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37740/0 (socket says 172.21.15.39:37740) 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.092+0000 7f13bd400640 1 -- 172.21.15.39:0/1925356885 learned_addr learned my addr 172.21.15.39:0/1925356885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13be800640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 0x7f13c01a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 -- 172.21.15.39:0/1925356885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 msgr2=0x7f13c0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 -- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 msgr2=0x7f13c01a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 0x7f13c01a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 -- 172.21.15.39:0/1925356885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c01a1770 con 0x7f13c0104ad0 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13be800640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 0x7f13c01a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bd400640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9f0 tx=0x7f13ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00cd80 con 0x7f13c0104ad0 2024-03-16T10:08:48.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ac010910 con 0x7f13c0104ad0 2024-03-16T10:08:48.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.093+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c01a1a60 con 0x7f13c0104ad0 2024-03-16T10:08:48.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.094+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac018990 con 0x7f13c0104ad0 2024-03-16T10:08:48.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.094+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c01a2000 con 0x7f13c0104ad0 2024-03-16T10:08:48.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.095+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c010ac00 con 0x7f13c0104ad0 2024-03-16T10:08:48.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.097+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f13ac010ac0 con 0x7f13c0104ad0 2024-03-16T10:08:48.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.099+0000 7f13b6a00640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 0x7f13a40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.099+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f13ac0cf1b0 con 0x7f13c0104ad0 2024-03-16T10:08:48.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.099+0000 7f13bde00640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 0x7f13a40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.100+0000 7f13bde00640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 0x7f13a40a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13c0104930 tx=0x7f13b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:48.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.107+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f13ac09be40 con 0x7f13c0104ad0 2024-03-16T10:08:48.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.324+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13c00623e0 con 0x7f13c0104ad0 2024-03-16T10:08:48.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.325+0000 7f13b6a00640 1 -- 172.21.15.39:0/1925356885 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f13ac09b740 con 0x7f13c0104ad0 2024-03-16T10:08:48.326 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 msgr2=0x7f13a40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 0x7f13a40a4d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13c0104930 tx=0x7f13b0005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 msgr2=0x7f13c019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00e9f0 tx=0x7f13ac00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 shutdown_connections 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13a40a2880 0x7f13a40a4d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13c0105490 0x7f13c01a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13c0104ad0 0x7f13c019a090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 --2- 172.21.15.39:0/1925356885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13c01038e0 0x7f13c0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 >> 172.21.15.39:0/1925356885 conn(0x7f13c00fee50 msgr2=0x7f13c0107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 shutdown_connections 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.330+0000 7f13bf200640 1 -- 172.21.15.39:0/1925356885 wait complete. 2024-03-16T10:08:48.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 -- 172.21.15.39:0/629966701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a4104af0 msgr2=0x7fe2a4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/629966701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a4104af0 0x7fe2a4104f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009900 tx=0x7fe28c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 -- 172.21.15.39:0/629966701 shutdown_connections 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/629966701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a41054b0 0x7fe2a410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/629966701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a4104af0 0x7fe2a4104f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/629966701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4103900 0x7fe2a4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 -- 172.21.15.39:0/629966701 >> 172.21.15.39:0/629966701 conn(0x7fe2a40fee90 msgr2=0x7fe2a41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 -- 172.21.15.39:0/629966701 shutdown_connections 2024-03-16T10:08:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.566+0000 7fe2a3e00640 1 -- 172.21.15.39:0/629966701 wait complete. 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.567+0000 7fe2a3e00640 1 Processor -- start 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.567+0000 7fe2a3e00640 1 -- start start 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4104af0 0x7fe2a419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410f580 con 0x7fe2a4104af0 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410f400 con 0x7fe2a41054b0 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410f700 con 0x7fe2a4103900 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe29be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4104af0 0x7fe2a419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44120/0 (socket says 172.21.15.39:44120) 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:37748/0 (socket says 172.21.15.39:37748) 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.568+0000 7fe2a3400640 1 -- 172.21.15.39:0/2846794181 learned_addr learned my addr 172.21.15.39:0/2846794181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 -- 172.21.15.39:0/2846794181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 msgr2=0x7fe2a41a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 -- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4104af0 msgr2=0x7fe2a419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4104af0 0x7fe2a419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 -- 172.21.15.39:0/2846794181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a41a17e0 con 0x7fe2a4103900 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a3400640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a2a00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe29400d6a0 tx=0x7fe29400db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294017070 con 0x7fe2a4103900 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a41a1ad0 con 0x7fe2a4103900 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294009d00 con 0x7fe2a4103900 2024-03-16T10:08:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294003c60 con 0x7fe2a4103900 2024-03-16T10:08:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.569+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a41a2040 con 0x7fe2a4103900 2024-03-16T10:08:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.570+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a410abc0 con 0x7fe2a4103900 2024-03-16T10:08:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.574+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe294009820 con 0x7fe2a4103900 2024-03-16T10:08:48.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.576+0000 7fe2a8a00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 0x7fe2880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:48.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.576+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fe2940ce6f0 con 0x7fe2a4103900 2024-03-16T10:08:48.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.576+0000 7fe29be00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 0x7fe2880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:48.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.577+0000 7fe29be00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 0x7fe2880a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe28c02c7b0 tx=0x7fe28c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.580+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe29409ad70 con 0x7fe2a4103900 2024-03-16T10:08:48.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:48 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1792264930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:48 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1925356885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:48 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1792264930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:48 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1925356885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:48.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.788+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe2a40623e0 con 0x7fe2a4103900 2024-03-16T10:08:48.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.788+0000 7fe2a8a00640 1 -- 172.21.15.39:0/2846794181 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fe29409ee10 con 0x7fe2a4103900 2024-03-16T10:08:48.790 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:08:48.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 msgr2=0x7fe2880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 0x7fe2880a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe28c02c7b0 tx=0x7fe28c005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 msgr2=0x7fe2a4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe29400d6a0 tx=0x7fe29400db60 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 shutdown_connections 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fe2880a2800 0x7fe2880a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a41054b0 0x7fe2a41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2a4104af0 0x7fe2a419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 --2- 172.21.15.39:0/2846794181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2a4103900 0x7fe2a4199bc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 >> 172.21.15.39:0/2846794181 conn(0x7fe2a40fee90 msgr2=0x7fe2a4107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 shutdown_connections 2024-03-16T10:08:48.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:48.791+0000 7fe2a3e00640 1 -- 172.21.15.39:0/2846794181 wait complete. 2024-03-16T10:08:48.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:08:48.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:48 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1792264930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:48.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:48 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1925356885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.027+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1411076022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 msgr2=0x7f5b9c0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.027+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1411076022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c0746c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009900 tx=0x7f5b8802bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1411076022 shutdown_connections 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1411076022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c1095a0 0x7f5b9c10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1411076022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c0746c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1411076022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c075820 0x7f5b9c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1411076022 >> 172.21.15.39:0/1411076022 conn(0x7f5b9c0fcc50 msgr2=0x7f5b9c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:49.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1411076022 shutdown_connections 2024-03-16T10:08:49.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.028+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1411076022 wait complete. 2024-03-16T10:08:49.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.029+0000 7f5ba2e00640 1 Processor -- start 2024-03-16T10:08:49.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.029+0000 7f5ba2e00640 1 -- start start 2024-03-16T10:08:49.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c075820 0x7f5b9c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c1095a0 0x7f5b9c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f520 con 0x7f5b9c1095a0 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f3a0 con 0x7f5b9c074240 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c10f6a0 con 0x7f5b9c075820 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44136/0 (socket says 172.21.15.39:44136) 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 -- 172.21.15.39:0/1002808696 learned_addr learned my addr 172.21.15.39:0/1002808696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1000640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c075820 0x7f5b9c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba2400640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c1095a0 0x7f5b9c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 -- 172.21.15.39:0/1002808696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c075820 msgr2=0x7f5b9c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c075820 0x7f5b9c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 -- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c1095a0 msgr2=0x7f5b9c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c1095a0 0x7f5b9c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.030+0000 7f5ba1a00640 1 -- 172.21.15.39:0/1002808696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1a1800 con 0x7f5b9c074240 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5ba1a00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e9f0 tx=0x7f5b9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:49.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000cd80 con 0x7f5b9c074240 2024-03-16T10:08:49.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1a1af0 con 0x7f5b9c074240 2024-03-16T10:08:49.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b90010910 con 0x7f5b9c074240 2024-03-16T10:08:49.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b900188f0 con 0x7f5b9c074240 2024-03-16T10:08:49.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.031+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1a2010 con 0x7f5b9c074240 2024-03-16T10:08:49.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.032+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c1082d0 con 0x7f5b9c074240 2024-03-16T10:08:49.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.034+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5b90010ac0 con 0x7f5b9c074240 2024-03-16T10:08:49.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.036+0000 7f5b9a000640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 0x7f5b840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.036+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f5b900cf210 con 0x7f5b9c074240 2024-03-16T10:08:49.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.036+0000 7f5ba1000640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 0x7f5b840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.037+0000 7f5ba1000640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 0x7f5b840a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c7b0 tx=0x7f5b88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:49.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.040+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b9009b850 con 0x7f5b9c074240 2024-03-16T10:08:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.240+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5b9c0623e0 con 0x7f5b9c074240 2024-03-16T10:08:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.241+0000 7f5b9a000640 1 -- 172.21.15.39:0/1002808696 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5b9009f8f0 con 0x7f5b9c074240 2024-03-16T10:08:49.242 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 msgr2=0x7f5b840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 0x7f5b840a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c7b0 tx=0x7f5b88005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 msgr2=0x7f5b9c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000e9f0 tx=0x7f5b9000eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 shutdown_connections 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f5b840a2800 0x7f5b840a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9c1095a0 0x7f5b9c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b9c075820 0x7f5b9c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 --2- 172.21.15.39:0/1002808696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9c074240 0x7f5b9c199be0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.243+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 >> 172.21.15.39:0/1002808696 conn(0x7f5b9c0fcc50 msgr2=0x7f5b9c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.244+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 shutdown_connections 2024-03-16T10:08:49.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.244+0000 7f5ba2e00640 1 -- 172.21.15.39:0/1002808696 wait complete. 2024-03-16T10:08:49.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:08:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 -- 172.21.15.39:0/2268938614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f4102070 msgr2=0x7f22f4104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 --2- 172.21.15.39:0/2268938614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f4102070 0x7f22f4104460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009ca0 tx=0x7f22e402c100 comp rx=0 tx=0).stop 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 -- 172.21.15.39:0/2268938614 shutdown_connections 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 --2- 172.21.15.39:0/2268938614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 0x7f22f4109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 --2- 172.21.15.39:0/2268938614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f41049a0 0x7f22f4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 --2- 172.21.15.39:0/2268938614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f4102070 0x7f22f4104460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 -- 172.21.15.39:0/2268938614 >> 172.21.15.39:0/2268938614 conn(0x7f22f40fba60 msgr2=0x7f22f40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 -- 172.21.15.39:0/2268938614 shutdown_connections 2024-03-16T10:08:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.487+0000 7f22f9000640 1 -- 172.21.15.39:0/2268938614 wait complete. 2024-03-16T10:08:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 Processor -- start 2024-03-16T10:08:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 -- start start 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f41049a0 0x7f22f419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 0x7f22f41a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410d270 con 0x7f22f41049a0 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410d0f0 con 0x7f22f4102070 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f9000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410d3f0 con 0x7f22f41072d0 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.489+0000 7f22f3400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:44156/0 (socket says 172.21.15.39:44156) 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 -- 172.21.15.39:0/4066792043 learned_addr learned my addr 172.21.15.39:0/4066792043 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f2a00640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f41049a0 0x7f22f419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3e00640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 0x7f22f41a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 -- 172.21.15.39:0/4066792043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 msgr2=0x7f22f41a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 0x7f22f41a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 -- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f41049a0 msgr2=0x7f22f419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f41049a0 0x7f22f419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 -- 172.21.15.39:0/4066792043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f41a1700 con 0x7f22f4102070 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f3400640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009dc0 tx=0x7f22e4035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4039040 con 0x7f22f4102070 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e4044070 con 0x7f22f4102070 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.490+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a1990 con 0x7f22f4102070 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.491+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4004460 con 0x7f22f4102070 2024-03-16T10:08:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.491+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a1e80 con 0x7f22f4102070 2024-03-16T10:08:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.492+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f4108e20 con 0x7f22f4102070 2024-03-16T10:08:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.493+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f22e4004600 con 0x7f22f4102070 2024-03-16T10:08:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.495+0000 7f22ebe00640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.495+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f22e40f14e0 con 0x7f22f4102070 2024-03-16T10:08:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.495+0000 7f22f2a00640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.496+0000 7f22f2a00640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 0x7f22d80a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f22f419aff0 tx=0x7f22dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.498+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22e40be170 con 0x7f22f4102070 2024-03-16T10:08:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:49 smithi194 ceph-mon[36445]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:49 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2846794181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:49 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1002808696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:49 smithi039 ceph-mon[31510]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:49 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2846794181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:49 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1002808696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.757+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f40623e0 con 0x7f22f4102070 2024-03-16T10:08:49.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.758+0000 7f22ebe00640 1 -- 172.21.15.39:0/4066792043 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f22e40bda70 con 0x7f22f4102070 2024-03-16T10:08:49.759 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:49.759 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:08:49.760 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.760+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 msgr2=0x7f22d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.760+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 0x7f22d80a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f22f419aff0 tx=0x7f22dc005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.760+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 msgr2=0x7f22f4199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.760+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009dc0 tx=0x7f22e4035d90 comp rx=0 tx=0).stop 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 shutdown_connections 2024-03-16T10:08:49.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22d80a2800 0x7f22d80a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22f41072d0 0x7f22f41a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22f41049a0 0x7f22f419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 --2- 172.21.15.39:0/4066792043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f4102070 0x7f22f4199ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 >> 172.21.15.39:0/4066792043 conn(0x7f22f40fba60 msgr2=0x7f22f4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 shutdown_connections 2024-03-16T10:08:49.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.761+0000 7f22f9000640 1 -- 172.21.15.39:0/4066792043 wait complete. 2024-03-16T10:08:49.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:49 smithi105 ceph-mon[36559]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:49 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2846794181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:49 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1002808696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:50.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 -- 172.21.15.39:0/2778043580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd360102070 msgr2=0x7fd360104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 --2- 172.21.15.39:0/2778043580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd360102070 0x7fd360104460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd350009900 tx=0x7fd35002bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:50.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 -- 172.21.15.39:0/2778043580 shutdown_connections 2024-03-16T10:08:50.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 --2- 172.21.15.39:0/2778043580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 --2- 172.21.15.39:0/2778043580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3601049a0 0x7fd360106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 --2- 172.21.15.39:0/2778043580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd360102070 0x7fd360104460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.998+0000 7fd35fe00640 1 -- 172.21.15.39:0/2778043580 >> 172.21.15.39:0/2778043580 conn(0x7fd3600fba60 msgr2=0x7fd3600fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.999+0000 7fd35fe00640 1 -- 172.21.15.39:0/2778043580 shutdown_connections 2024-03-16T10:08:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:49.999+0000 7fd35fe00640 1 -- 172.21.15.39:0/2778043580 wait complete. 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 Processor -- start 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 -- start start 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 0x7fd3601019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3601049a0 0x7fd360100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010d280 con 0x7fd360102070 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010d100 con 0x7fd3601072d0 2024-03-16T10:08:50.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35fe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010d400 con 0x7fd3601049a0 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 0x7fd3601019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 0x7fd3601019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36580/0 (socket says 172.21.15.39:36580) 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35ea00640 1 -- 172.21.15.39:0/3874300229 learned_addr learned my addr 172.21.15.39:0/3874300229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.000+0000 7fd35e000640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3601049a0 0x7fd360100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 -- 172.21.15.39:0/3874300229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3601049a0 msgr2=0x7fd360100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3601049a0 0x7fd360100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 -- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 msgr2=0x7fd3601019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 0x7fd3601019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 -- 172.21.15.39:0/3874300229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd360100df0 con 0x7fd3601072d0 2024-03-16T10:08:50.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35f400640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360100560 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd358009fd0 tx=0x7fd35800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ce20 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd358005020 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601a19c0 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.001+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358010720 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.002+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a1f10 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.003+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3600fd550 con 0x7fd3601072d0 2024-03-16T10:08:50.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.004+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fd358005540 con 0x7fd3601072d0 2024-03-16T10:08:50.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.007+0000 7fd357400640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 0x7fd3440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.007+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fd3580cf460 con 0x7fd3601072d0 2024-03-16T10:08:50.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.007+0000 7fd35ea00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 0x7fd3440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.008+0000 7fd35ea00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 0x7fd3440a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd350000c00 tx=0x7fd35002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.010+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd35809c130 con 0x7fd3601072d0 2024-03-16T10:08:50.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.215+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3600623e0 con 0x7fd3601072d0 2024-03-16T10:08:50.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.217+0000 7fd357400640 1 -- 172.21.15.39:0/3874300229 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24850 (secure 0 0 0) 0x7fd35809ba30 con 0x7fd3601072d0 2024-03-16T10:08:50.219 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:50.219 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:08:14.004515+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.219+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 msgr2=0x7fd3440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.219+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 0x7fd3440a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd350000c00 tx=0x7fd35002c880 comp rx=0 tx=0).stop 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 msgr2=0x7fd360100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360100560 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd358009fd0 tx=0x7fd35800ef10 comp rx=0 tx=0).stop 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 shutdown_connections 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fd3440a2800 0x7fd3440a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3601072d0 0x7fd360100560 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3601049a0 0x7fd360100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 --2- 172.21.15.39:0/3874300229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd360102070 0x7fd3601019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 >> 172.21.15.39:0/3874300229 conn(0x7fd3600fba60 msgr2=0x7fd360102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 shutdown_connections 2024-03-16T10:08:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.220+0000 7fd35fe00640 1 -- 172.21.15.39:0/3874300229 wait complete. 2024-03-16T10:08:50.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:08:50.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:08:50.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 -- 172.21.15.39:0/786302306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c107390 msgr2=0x7f557c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 --2- 172.21.15.39:0/786302306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c107390 0x7f557c1097e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ca0 tx=0x7f557402c0f0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 -- 172.21.15.39:0/786302306 shutdown_connections 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 --2- 172.21.15.39:0/786302306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c107390 0x7f557c1097e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 --2- 172.21.15.39:0/786302306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c0ffd00 0x7f557c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 --2- 172.21.15.39:0/786302306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ff310 0x7f557c0ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 -- 172.21.15.39:0/786302306 >> 172.21.15.39:0/786302306 conn(0x7f557c0fae60 msgr2=0x7f557c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 -- 172.21.15.39:0/786302306 shutdown_connections 2024-03-16T10:08:50.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.463+0000 7f5581e00640 1 -- 172.21.15.39:0/786302306 wait complete. 2024-03-16T10:08:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.464+0000 7f5581e00640 1 Processor -- start 2024-03-16T10:08:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.464+0000 7f5581e00640 1 -- start start 2024-03-16T10:08:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c107390 0x7f557c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10d330 con 0x7f557c107390 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10d1b0 con 0x7f557c0ffd00 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10d4b0 con 0x7f557c0ff310 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5580a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5580a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41220/0 (socket says 172.21.15.39:41220) 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54482/0 (socket says 172.21.15.39:54482) 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5580a00640 1 -- 172.21.15.39:0/3199323293 learned_addr learned my addr 172.21.15.39:0/3199323293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5581400640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c107390 0x7f557c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 -- 172.21.15.39:0/3199323293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 msgr2=0x7f557c199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 -- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c107390 msgr2=0x7f557c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c107390 0x7f557c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f557be00640 1 -- 172.21.15.39:0/3199323293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1a1760 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.465+0000 7f5580a00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f557be00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9f0 tx=0x7f556800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556800ccc0 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568010910 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1a1a50 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568018960 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.466+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1a1ff0 con 0x7f557c0ffd00 2024-03-16T10:08:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.467+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c1a2220 con 0x7f557c0ffd00 2024-03-16T10:08:50.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.470+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5568010ac0 con 0x7f557c0ffd00 2024-03-16T10:08:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.471+0000 7f5579600640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 0x7f55640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.471+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f55680cf140 con 0x7f557c0ffd00 2024-03-16T10:08:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.472+0000 7f5580a00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 0x7f55640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.472+0000 7f5580a00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 0x7f55640a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5570011500 tx=0x7f5570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.479+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f556809b7c0 con 0x7f557c0ffd00 2024-03-16T10:08:50.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:50 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4066792043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:50 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3874300229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.680+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f557c1a2860 con 0x7f557c0ffd00 2024-03-16T10:08:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.680+0000 7f5579600640 1 -- 172.21.15.39:0/3199323293 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f556809f860 con 0x7f557c0ffd00 2024-03-16T10:08:50.682 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:08:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 msgr2=0x7f55640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 0x7f55640a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5570011500 tx=0x7f5570005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 msgr2=0x7f557c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9f0 tx=0x7f556800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 shutdown_connections 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f55640a2880 0x7f55640a4d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f557c107390 0x7f557c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f557c0ffd00 0x7f557c19a0d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 --2- 172.21.15.39:0/3199323293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f557c0ff310 0x7f557c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 >> 172.21.15.39:0/3199323293 conn(0x7f557c0fae60 msgr2=0x7f557c0fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 shutdown_connections 2024-03-16T10:08:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.683+0000 7f5581e00640 1 -- 172.21.15.39:0/3199323293 wait complete. 2024-03-16T10:08:50.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:08:50.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:50 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4066792043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:50 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3874300229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:50 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4066792043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:50 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3874300229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.921+0000 7f76bce00640 1 -- 172.21.15.39:0/1619392403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8105490 msgr2=0x7f76b810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 --2- 172.21.15.39:0/1619392403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8105490 0x7f76b810bd20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009ca0 tx=0x7f76b002c0f0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 -- 172.21.15.39:0/1619392403 shutdown_connections 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 --2- 172.21.15.39:0/1619392403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8105490 0x7f76b810bd20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 --2- 172.21.15.39:0/1619392403 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b8104ad0 0x7f76b8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 --2- 172.21.15.39:0/1619392403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b81038e0 0x7f76b8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 -- 172.21.15.39:0/1619392403 >> 172.21.15.39:0/1619392403 conn(0x7f76b80fee50 msgr2=0x7f76b8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 -- 172.21.15.39:0/1619392403 shutdown_connections 2024-03-16T10:08:50.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.922+0000 7f76bce00640 1 -- 172.21.15.39:0/1619392403 wait complete. 2024-03-16T10:08:50.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 Processor -- start 2024-03-16T10:08:50.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 -- start start 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 0x7f76b8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 0x7f76b819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8105490 0x7f76b81a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810f540 con 0x7f76b8105490 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810f3c0 con 0x7f76b8104ad0 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76bce00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810f6c0 con 0x7f76b81038e0 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.923+0000 7f76b7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 0x7f76b8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 0x7f76b819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76b7400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 0x7f76b8199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41234/0 (socket says 172.21.15.39:41234) 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76b7400640 1 -- 172.21.15.39:0/4235105348 learned_addr learned my addr 172.21.15.39:0/4235105348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76b7e00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8105490 0x7f76b81a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 -- 172.21.15.39:0/4235105348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 msgr2=0x7f76b8199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 0x7f76b8199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 -- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8105490 msgr2=0x7f76b81a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8105490 0x7f76b81a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 -- 172.21.15.39:0/4235105348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81a1770 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76afe00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 0x7f76b819a090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a000ccc0 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a0010910 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.924+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81a1a60 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.925+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0018960 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.925+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a2000 con 0x7f76b8104ad0 2024-03-16T10:08:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.926+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b810ac00 con 0x7f76b8104ad0 2024-03-16T10:08:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.927+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f76a0010ac0 con 0x7f76b8104ad0 2024-03-16T10:08:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.929+0000 7f76b4c00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 0x7f769c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.929+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f76a00cf1d0 con 0x7f76b8104ad0 2024-03-16T10:08:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.929+0000 7f76b7400640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 0x7f769c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:50.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.930+0000 7f76b7400640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 0x7f769c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76a80058c0 tx=0x7f76a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:50.933+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76a009bea0 con 0x7f76b8104ad0 2024-03-16T10:08:51.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.139+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76b80623e0 con 0x7f76b8104ad0 2024-03-16T10:08:51.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.139+0000 7f76b4c00640 1 -- 172.21.15.39:0/4235105348 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f76a009b7a0 con 0x7f76b8104ad0 2024-03-16T10:08:51.141 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 msgr2=0x7f769c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 0x7f769c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76a80058c0 tx=0x7f76a8005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 msgr2=0x7f76b819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 0x7f76b819a090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9f0 tx=0x7f76a000eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 shutdown_connections 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f769c0a2880 0x7f769c0a4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8105490 0x7f76b81a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76b8104ad0 0x7f76b819a090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 --2- 172.21.15.39:0/4235105348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76b81038e0 0x7f76b8199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 >> 172.21.15.39:0/4235105348 conn(0x7f76b80fee50 msgr2=0x7f76b8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 shutdown_connections 2024-03-16T10:08:51.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.142+0000 7f76bce00640 1 -- 172.21.15.39:0/4235105348 wait complete. 2024-03-16T10:08:51.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:08:51.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 -- 172.21.15.39:0/1800299612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 msgr2=0x7f07ac101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 --2- 172.21.15.39:0/1800299612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac101b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f079c009ca0 tx=0x7f079c02c0f0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 -- 172.21.15.39:0/1800299612 shutdown_connections 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 --2- 172.21.15.39:0/1800299612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac103290 0x7f07ac109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 --2- 172.21.15.39:0/1800299612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1028d0 0x7f07ac102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 --2- 172.21.15.39:0/1800299612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac101b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 -- 172.21.15.39:0/1800299612 >> 172.21.15.39:0/1800299612 conn(0x7f07ac0fcc50 msgr2=0x7f07ac0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 -- 172.21.15.39:0/1800299612 shutdown_connections 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.378+0000 7f07abe00640 1 -- 172.21.15.39:0/1800299612 wait complete. 2024-03-16T10:08:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.379+0000 7f07abe00640 1 Processor -- start 2024-03-16T10:08:51.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 -- start start 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 0x7f07ac073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 0x7f07ac074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10d350 con 0x7f07ac1028d0 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10d1d0 con 0x7f07ac1016e0 2024-03-16T10:08:51.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07abe00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10d4d0 con 0x7f07ac103290 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07aaa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54508/0 (socket says 172.21.15.39:54508) 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.380+0000 7f07aaa00640 1 -- 172.21.15.39:0/1850363543 learned_addr learned my addr 172.21.15.39:0/1850363543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07ab400640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 0x7f07ac074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aa000640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 0x7f07ac073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 -- 172.21.15.39:0/1850363543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 msgr2=0x7f07ac074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 0x7f07ac074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 -- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 msgr2=0x7f07ac073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 0x7f07ac073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 -- 172.21.15.39:0/1850363543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac0749a0 con 0x7f07ac1016e0 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07ab400640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 0x7f07ac074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aa000640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 0x7f07ac073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.381+0000 7f07aaa00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f079c0324c0 tx=0x7f079c032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.382+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c041020 con 0x7f07ac1016e0 2024-03-16T10:08:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.382+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f079c034cd0 con 0x7f07ac1016e0 2024-03-16T10:08:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.382+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac1a1ae0 con 0x7f07ac1016e0 2024-03-16T10:08:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.382+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c039040 con 0x7f07ac1016e0 2024-03-16T10:08:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.382+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac1a2080 con 0x7f07ac1016e0 2024-03-16T10:08:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.383+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac108aa0 con 0x7f07ac1016e0 2024-03-16T10:08:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.385+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f079c044020 con 0x7f07ac1016e0 2024-03-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.386+0000 7f07b0a00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 0x7f07900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.386+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f079c0f1120 con 0x7f07ac1016e0 2024-03-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.387+0000 7f07aa000640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 0x7f07900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.387+0000 7f07aa000640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 0x7f07900a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07ac075300 tx=0x7f0798005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.393+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f079c04b020 con 0x7f07ac1016e0 2024-03-16T10:08:51.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.613+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f07ac02d400 con 0x7f07ac1016e0 2024-03-16T10:08:51.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.614+0000 7f07b0a00640 1 -- 172.21.15.39:0/1850363543 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f079c0bdf60 con 0x7f07ac1016e0 2024-03-16T10:08:51.615 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:08:51.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 msgr2=0x7f07900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 0x7f07900a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07ac075300 tx=0x7f0798005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 msgr2=0x7f07ac075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f079c0324c0 tx=0x7f079c032f40 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 shutdown_connections 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f07900a2880 0x7f07900a4d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac103290 0x7f07ac074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07ac1028d0 0x7f07ac073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 --2- 172.21.15.39:0/1850363543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07ac1016e0 0x7f07ac075590 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 >> 172.21.15.39:0/1850363543 conn(0x7f07ac0fcc50 msgr2=0x7f07ac105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 shutdown_connections 2024-03-16T10:08:51.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.616+0000 7f07abe00640 1 -- 172.21.15.39:0/1850363543 wait complete. 2024-03-16T10:08:51.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:08:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:51 smithi194 ceph-mon[36445]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3199323293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4235105348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:51.710 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:51 smithi039 ceph-mon[31510]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:51.710 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3199323293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:51.711 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4235105348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 -- 172.21.15.39:0/2130124229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 msgr2=0x7f776c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 --2- 172.21.15.39:0/2130124229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c109b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7764009ca0 tx=0x7f776402c100 comp rx=0 tx=0).stop 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 -- 172.21.15.39:0/2130124229 shutdown_connections 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 --2- 172.21.15.39:0/2130124229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c109b20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 --2- 172.21.15.39:0/2130124229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1028d0 0x7f776c102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 --2- 172.21.15.39:0/2130124229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1016e0 0x7f776c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 -- 172.21.15.39:0/2130124229 >> 172.21.15.39:0/2130124229 conn(0x7f776c0fcc50 msgr2=0x7f776c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 -- 172.21.15.39:0/2130124229 shutdown_connections 2024-03-16T10:08:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.851+0000 7f7770a00640 1 -- 172.21.15.39:0/2130124229 wait complete. 2024-03-16T10:08:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.852+0000 7f7770a00640 1 Processor -- start 2024-03-16T10:08:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.852+0000 7f7770a00640 1 -- start start 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1016e0 0x7f776c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 0x7f776c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d360 con 0x7f776c1016e0 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d1e0 con 0x7f776c103290 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f7770a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f776c10d4e0 con 0x7f776c1028d0 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1016e0 0x7f776c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 0x7f776c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 0x7f776c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41270/0 (socket says 172.21.15.39:41270) 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776aa00640 1 -- 172.21.15.39:0/3941405185 learned_addr learned my addr 172.21.15.39:0/3941405185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 -- 172.21.15.39:0/3941405185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 msgr2=0x7f776c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 0x7f776c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 -- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1016e0 msgr2=0x7f776c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1016e0 0x7f776c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.853+0000 7f776be00640 1 -- 172.21.15.39:0/3941405185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c19f560 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f776be00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c19eef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f776402c640 tx=0x7f7764035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764039040 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7764044070 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640044a0 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c19f7f0 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.854+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c19fc60 con 0x7f776c103290 2024-03-16T10:08:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.855+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c108a00 con 0x7f776c103290 2024-03-16T10:08:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.858+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7764004640 con 0x7f776c103290 2024-03-16T10:08:51.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.859+0000 7f7763e00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 0x7f77500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:51.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.859+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f77640f12d0 con 0x7f776c103290 2024-03-16T10:08:51.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.859+0000 7f776b400640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 0x7f77500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.860+0000 7f776b400640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 0x7f77500a4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f776c102730 tx=0x7f775c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:51.863+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f776404e020 con 0x7f776c103290 2024-03-16T10:08:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:51 smithi105 ceph-mon[36559]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-03-16T10:08:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3199323293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:08:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4235105348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.065+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f776c0623e0 con 0x7f776c103290 2024-03-16T10:08:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.065+0000 7f7763e00640 1 -- 172.21.15.39:0/3941405185 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f77640bdf60 con 0x7f776c103290 2024-03-16T10:08:52.067 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 msgr2=0x7f77500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 0x7f77500a4d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f776c102730 tx=0x7f775c005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 msgr2=0x7f776c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c19eef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f776402c640 tx=0x7f7764035d90 comp rx=0 tx=0).stop 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 shutdown_connections 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f77500a2880 0x7f77500a4d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f776c103290 0x7f776c19eef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f776c1028d0 0x7f776c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 --2- 172.21.15.39:0/3941405185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f776c1016e0 0x7f776c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 >> 172.21.15.39:0/3941405185 conn(0x7f776c0fcc50 msgr2=0x7f776c105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 shutdown_connections 2024-03-16T10:08:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.069+0000 7f7770a00640 1 -- 172.21.15.39:0/3941405185 wait complete. 2024-03-16T10:08:52.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.312+0000 7f0824a00640 1 -- 172.21.15.39:0/3752462319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201016c0 msgr2=0x7f0820101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.312+0000 7f0824a00640 1 --2- 172.21.15.39:0/3752462319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201016c0 0x7f0820101b40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.312+0000 7f0824a00640 1 -- 172.21.15.39:0/3752462319 shutdown_connections 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 --2- 172.21.15.39:0/3752462319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 0x7f0820108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 --2- 172.21.15.39:0/3752462319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201016c0 0x7f0820101b40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 --2- 172.21.15.39:0/3752462319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201004d0 0x7f08201008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 -- 172.21.15.39:0/3752462319 >> 172.21.15.39:0/3752462319 conn(0x7f08200fba60 msgr2=0x7f08200fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:52.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 -- 172.21.15.39:0/3752462319 shutdown_connections 2024-03-16T10:08:52.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.313+0000 7f0824a00640 1 -- 172.21.15.39:0/3752462319 wait complete. 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 Processor -- start 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 -- start start 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201016c0 0x7f082019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 0x7f08201a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f082010c160 con 0x7f08201016c0 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f082010bfe0 con 0x7f08201004d0 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f0824a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f082010c2e0 con 0x7f0820102080 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f081f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f081ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201016c0 0x7f082019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f081f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54552/0 (socket says 172.21.15.39:54552) 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f081f400640 1 -- 172.21.15.39:0/2659952109 learned_addr learned my addr 172.21.15.39:0/2659952109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.314+0000 7f081fe00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 0x7f08201a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 -- 172.21.15.39:0/2659952109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 msgr2=0x7f08201a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 0x7f08201a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 -- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201016c0 msgr2=0x7f082019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201016c0 0x7f082019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 -- 172.21.15.39:0/2659952109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08201a1720 con 0x7f08201004d0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f081f400640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f081000ebc0 tx=0x7f081000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810003970 con 0x7f08201004d0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08201a1a10 con 0x7f08201004d0 2024-03-16T10:08:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810004030 con 0x7f08201004d0 2024-03-16T10:08:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.315+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100197b0 con 0x7f08201004d0 2024-03-16T10:08:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.316+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08201a1f30 con 0x7f08201004d0 2024-03-16T10:08:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.317+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08201a21c0 con 0x7f08201004d0 2024-03-16T10:08:52.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.319+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f08100041e0 con 0x7f08201004d0 2024-03-16T10:08:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.320+0000 7f0817e00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 0x7f08040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.320+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f08100cf790 con 0x7f08201004d0 2024-03-16T10:08:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.320+0000 7f081ea00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 0x7f08040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.321+0000 7f081ea00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 0x7f08040a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f080802c7c0 tx=0x7f0808005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.323+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f081009c420 con 0x7f08201004d0 2024-03-16T10:08:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.582+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08201a2390 con 0x7f08201004d0 2024-03-16T10:08:52.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.583+0000 7f0817e00640 1 -- 172.21.15.39:0/2659952109 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f081009bd20 con 0x7f08201004d0 2024-03-16T10:08:52.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:52.585 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:06:40.587364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24557,24569,24518,24523,24544]},"id":1}]} 2024-03-16T10:08:52.585 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 msgr2=0x7f08040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 0x7f08040a4cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f080802c7c0 tx=0x7f0808005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 msgr2=0x7f0820199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f081000ebc0 tx=0x7f081000c2d0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 shutdown_connections 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f08040a2800 0x7f08040a4cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0820102080 0x7f08201a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08201016c0 0x7f082019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 --2- 172.21.15.39:0/2659952109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08201004d0 0x7f0820199b70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 >> 172.21.15.39:0/2659952109 conn(0x7f08200fba60 msgr2=0x7f08200fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:52.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 shutdown_connections 2024-03-16T10:08:52.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.586+0000 7f0824a00640 1 -- 172.21.15.39:0/2659952109 wait complete. 2024-03-16T10:08:52.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:08:52.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1850363543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3941405185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 -- 172.21.15.39:0/3410362279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c101160 msgr2=0x7f9f1c1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 --2- 172.21.15.39:0/3410362279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c101160 0x7f9f1c1015e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009ca0 tx=0x7f9f1402c0f0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 -- 172.21.15.39:0/3410362279 shutdown_connections 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 --2- 172.21.15.39:0/3410362279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c101160 0x7f9f1c1015e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 --2- 172.21.15.39:0/3410362279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c1007c0 0x7f9f1c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 --2- 172.21.15.39:0/3410362279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 -- 172.21.15.39:0/3410362279 >> 172.21.15.39:0/3410362279 conn(0x7f9f1c0fba60 msgr2=0x7f9f1c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.825+0000 7f9f20e00640 1 -- 172.21.15.39:0/3410362279 shutdown_connections 2024-03-16T10:08:52.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.826+0000 7f9f20e00640 1 -- 172.21.15.39:0/3410362279 wait complete. 2024-03-16T10:08:52.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.827+0000 7f9f20e00640 1 Processor -- start 2024-03-16T10:08:52.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.827+0000 7f9f20e00640 1 -- start start 2024-03-16T10:08:52.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c1007c0 0x7f9f1c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c101160 0x7f9f1c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10f440 con 0x7f9f1c101160 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10f2c0 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f20e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c10f5c0 con 0x7f9f1c1007c0 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54580/0 (socket says 172.21.15.39:54580) 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 -- 172.21.15.39:0/2255769398 learned_addr learned my addr 172.21.15.39:0/2255769398 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1aa00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c1007c0 0x7f9f1c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1be00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c101160 0x7f9f1c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 -- 172.21.15.39:0/2255769398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c1007c0 msgr2=0x7f9f1c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c1007c0 0x7f9f1c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 -- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c101160 msgr2=0x7f9f1c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c101160 0x7f9f1c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.828+0000 7f9f1b400640 1 -- 172.21.15.39:0/2255769398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c1a1640 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f1b400640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9f0 tx=0x7f9f0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:52.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00cd80 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f0c010910 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c018990 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c1a1930 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.829+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c1a1e30 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.830+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c0684f0 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.832+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f9f0c010ac0 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.833+0000 7f9f13e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 0x7f9f000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:52.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.834+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f9f0c0cf120 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.834+0000 7f9f1aa00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 0x7f9f000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:52.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.834+0000 7f9f1aa00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 0x7f9f000a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c19af30 tx=0x7f9f08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:52.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:52.837+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f0c09b760 con 0x7f9f1c0ffe60 2024-03-16T10:08:52.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1850363543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3941405185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1850363543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:08:52.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3941405185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:08:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.052+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f1c0624b0 con 0x7f9f1c0ffe60 2024-03-16T10:08:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.055+0000 7f9f13e00640 1 -- 172.21.15.39:0/2255769398 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24850 (secure 0 0 0) 0x7f9f0c0bb020 con 0x7f9f1c0ffe60 2024-03-16T10:08:53.057 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:08:53.057 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:08:14.004515+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:08:53.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 msgr2=0x7f9f000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 0x7f9f000a4d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c19af30 tx=0x7f9f08005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:53.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 msgr2=0x7f9f1c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00e9f0 tx=0x7f9f0c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 shutdown_connections 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f9f000a2880 0x7f9f000a4d40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f1c101160 0x7f9f1c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f1c1007c0 0x7f9f1c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 --2- 172.21.15.39:0/2255769398 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f1c0ffe60 0x7f9f1c199a20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 >> 172.21.15.39:0/2255769398 conn(0x7f9f1c0fba60 msgr2=0x7f9f1c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 shutdown_connections 2024-03-16T10:08:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.058+0000 7f9f20e00640 1 -- 172.21.15.39:0/2255769398 wait complete. 2024-03-16T10:08:53.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T10:06:03.734324+0000', 'modified': '2024-03-16T10:06:40.587364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24544, 'mds_3': 24523, 'mds_4': 24518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6833/1279561047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6832', 'nonce': 1279561047}, {'type': 'v1', 'addr': '172.21.15.194:6833', 'nonce': 1279561047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/1951962450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1951962450}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1951962450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/2605661802', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 2605661802}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 2605661802}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6835/106855660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 106855660}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 106855660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/3003167044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 3003167044}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 3003167044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.105:6837/3830303638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 3830303638}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 3830303638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.194:6837/3973444513', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 3973444513}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 3973444513}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.194:6839/985236448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 985236448}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 985236448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6841/1425412361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 1425412361}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 1425412361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6839/2189492991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 2189492991}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 2189492991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24557, 24569, 24518, 24523, 24544]} 2024-03-16T10:08:53.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-16T10:08:53.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 -- 172.21.15.39:0/1633347616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70104c30 msgr2=0x7f3a70107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 --2- 172.21.15.39:0/1633347616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70104c30 0x7f3a70107020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009900 tx=0x7f3a5c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 -- 172.21.15.39:0/1633347616 shutdown_connections 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 --2- 172.21.15.39:0/1633347616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70107560 0x7f3a70109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 --2- 172.21.15.39:0/1633347616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70104c30 0x7f3a70107020 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 --2- 172.21.15.39:0/1633347616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70102300 0x7f3a701046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.280+0000 7f3a76200640 1 -- 172.21.15.39:0/1633347616 >> 172.21.15.39:0/1633347616 conn(0x7f3a700fbc70 msgr2=0x7f3a700fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.281+0000 7f3a76200640 1 -- 172.21.15.39:0/1633347616 shutdown_connections 2024-03-16T10:08:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.281+0000 7f3a76200640 1 -- 172.21.15.39:0/1633347616 wait complete. 2024-03-16T10:08:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.282+0000 7f3a76200640 1 Processor -- start 2024-03-16T10:08:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.282+0000 7f3a76200640 1 -- start start 2024-03-16T10:08:53.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70104c30 0x7f3a701980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70107560 0x7f3a7019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010d450 con 0x7f3a70104c30 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010d2d0 con 0x7f3a70107560 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a76200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010d5d0 con 0x7f3a70102300 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a6f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a6f400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41322/0 (socket says 172.21.15.39:41322) 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a6f400640 1 -- 172.21.15.39:0/2824300649 learned_addr learned my addr 172.21.15.39:0/2824300649 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70107560 0x7f3a7019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.283+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70104c30 0x7f3a701980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 -- 172.21.15.39:0/2824300649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70107560 msgr2=0x7f3a7019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70107560 0x7f3a7019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 -- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70104c30 msgr2=0x7f3a701980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70104c30 0x7f3a701980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 -- 172.21.15.39:0/2824300649 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019f7d0 con 0x7f3a70102300 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a6f400640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e9f0 tx=0x7f3a6000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a67e00640 1 -- 172.21.15.39:0/2824300649 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000cd80 con 0x7f3a70102300 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a67e00640 1 -- 172.21.15.39:0/2824300649 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a60005500 con 0x7f3a70102300 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7019fac0 con 0x7f3a70102300 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a67e00640 1 -- 172.21.15.39:0/2824300649 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60010690 con 0x7f3a70102300 2024-03-16T10:08:53.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.284+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7019ff90 con 0x7f3a70102300 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.289+0000 7f3a78bfd740 1 Processor -- start 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.289+0000 7f3a78bfd740 1 -- start start 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.289+0000 7f3a67e00640 1 -- 172.21.15.39:0/2824300649 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3a600046c0 con 0x7f3a70102300 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf126650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf129150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cabf085c50 con 0x55cabf121b50 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cabf02e9d0 con 0x55cabf126cc0 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cabf081cf0 con 0x55cabf121160 2024-03-16T10:08:53.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41326/0 (socket says 172.21.15.39:41326) 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 -- 172.21.15.39:0/1060443143 learned_addr learned my addr 172.21.15.39:0/1060443143 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf129150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6f400640 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf126650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 -- 172.21.15.39:0/1060443143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 msgr2=0x55cabf129150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf129150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 -- 172.21.15.39:0/1060443143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 msgr2=0x55cabf126650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.290+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf126650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a6fe00640 1 -- 172.21.15.39:0/1060443143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabf048f10 con 0x55cabf121160 2024-03-16T10:08:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a7d8878852a4ed server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a5be00640 1 -- 172.21.15.39:0/1060443143 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68004d20 con 0x55cabf121160 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a5be00640 1 -- 172.21.15.39:0/1060443143 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a68004050 con 0x55cabf121160 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a5be00640 1 -- 172.21.15.39:0/1060443143 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68005220 con 0x55cabf121160 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.291+0000 7f3a67e00640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 0x7f3a540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- 172.21.15.39:0/1060443143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 msgr2=0x55cabf121580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a680118d0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a67e00640 1 -- 172.21.15.39:0/2824300649 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f3a600cf220 con 0x7f3a70102300 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a6f400640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 0x7f3a540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- 172.21.15.39:0/1060443143 shutdown_connections 2024-03-16T10:08:53.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf129150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf126650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/1060443143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf121580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- 172.21.15.39:0/1060443143 >> 172.21.15.39:0/1060443143 conn(0x55cabf11c6d0 msgr2=0x55cabf11cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- 172.21.15.39:0/1060443143 shutdown_connections 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- 172.21.15.39:0/1060443143 wait complete. 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a78bfd740 1 -- start start 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.292+0000 7f3a6f400640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 0x7f3a540a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a6009b780 tx=0x7f3a60004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 Processor -- start 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf12b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf1327a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cabf129c70 con 0x55cabf121b50 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cabf041e20 con 0x55cabf126cc0 2024-03-16T10:08:53.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cabf129df0 con 0x55cabf121160 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a78bfd740 10 client.0 _ll_register_callbacks cb 0x55cabf0417f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.293+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41334/0 (socket says 172.21.15.39:41334) 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 -- 172.21.15.39:0/4204777462 learned_addr learned my addr 172.21.15.39:0/4204777462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf12b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf1327a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 -- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 msgr2=0x55cabf1327a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf1327a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 -- 172.21.15.39:0/4204777462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 msgr2=0x55cabf12b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf12b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 -- 172.21.15.39:0/4204777462 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabed9a2d0 con 0x55cabf121160 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009de0 tx=0x7f3a68009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a93e0c05ad7609 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a58c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:53.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.294+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800f4e0 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f3a6800f680 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a68004050 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a68034050 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a78bfd740 10 client.25360 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cabee646b0 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a68004050 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3a68025040 con 0x55cabf121160 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.295+0000 7f3a78bfd740 10 client.25360 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a78bfd740 10 client.25360 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a78bfd740 20 client.25360 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf137ce0 0x55cabf139150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a78bfd740 4 client.25360 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:53.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 --> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cabf0e32b0 con 0x55cabf137ce0 2024-03-16T10:08:53.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf137ce0 0x55cabf139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf137ce0 0x55cabf139150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=948d9e68e901e946 server_cookie=154cf7108aad33c5 in_seq=0 out_seq=0 2024-03-16T10:08:53.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.296+0000 7f3a58c00640 10 client.25360 ms_handle_connect on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a58c00640 1 -- 172.21.15.39:0/4204777462 <== mds.4 v2:172.21.15.105:6836/3830303638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cabf0e32b0 con 0x55cabf137ce0 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a58c00640 10 client.25360 handle_command_reply: tid=0 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 client.25360 shutdown 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 msgr2=0x55cabf12b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009de0 tx=0x7f3a68009ac0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 shutdown_connections 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf137ce0 0x55cabf139150 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf1327a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf12b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a58c00640 0 client.25360 ms_handle_reset on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:53.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/4204777462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf12b220 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.297+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 >> 172.21.15.39:0/4204777462 conn(0x55cabf11c6d0 msgr2=0x55cabf11ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.298+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 shutdown_connections 2024-03-16T10:08:53.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.298+0000 7f3a78bfd740 1 -- 172.21.15.39:0/4204777462 wait complete. 2024-03-16T10:08:53.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.298+0000 7f3a78bfd740 20 client.25360 trim_cache size 0 max 16384 2024-03-16T10:08:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 Processor -- start 2024-03-16T10:08:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 -- start start 2024-03-16T10:08:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf132760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf12b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cabf137490 con 0x55cabf121b50 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cabf137b60 con 0x55cabf126cc0 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cabf138230 con 0x55cabf121160 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41338/0 (socket says 172.21.15.39:41338) 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.318+0000 7f3a6fe00640 1 -- 172.21.15.39:0/3529790584 learned_addr learned my addr 172.21.15.39:0/3529790584 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf12b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf132760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 -- 172.21.15.39:0/3529790584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 msgr2=0x55cabf12b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf12b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 -- 172.21.15.39:0/3529790584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 msgr2=0x55cabf132760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf132760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 -- 172.21.15.39:0/3529790584 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabf0f2750 con 0x55cabf121160 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a680194d0 tx=0x7f3a68009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e1ddbddfc4c484 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a5b400640 1 -- 172.21.15.39:0/3529790584 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68004000 con 0x55cabf121160 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a5b400640 1 -- 172.21.15.39:0/3529790584 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a680047f0 con 0x55cabf121160 2024-03-16T10:08:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.319+0000 7f3a5b400640 1 -- 172.21.15.39:0/3529790584 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68019bb0 con 0x55cabf121160 2024-03-16T10:08:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 -- 172.21.15.39:0/3529790584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 msgr2=0x55cabf11cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a680194d0 tx=0x7f3a68009a50 comp rx=0 tx=0).stop 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 -- 172.21.15.39:0/3529790584 shutdown_connections 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf126cc0 0x55cabf12b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121b50 0x55cabf132760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/3529790584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf121160 0x55cabf11cab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 -- 172.21.15.39:0/3529790584 >> 172.21.15.39:0/3529790584 conn(0x55cabf11c6d0 msgr2=0x55cabf12ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 -- 172.21.15.39:0/3529790584 shutdown_connections 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.320+0000 7f3a78bfd740 1 -- 172.21.15.39:0/3529790584 wait complete. 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 -- start start 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 Processor -- start 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121160 0x55cabf12e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 0x55cabf131a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cabf138fd0 con 0x55cabf121160 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cabf138900 con 0x55cabf121b50 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cabf1396a0 con 0x55cabf126cc0 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a78bfd740 10 client.0 _ll_register_callbacks cb 0x7f3a68004220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.321+0000 7f3a6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54612/0 (socket says 172.21.15.39:54612) 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 -- 172.21.15.39:0/2578382696 learned_addr learned my addr 172.21.15.39:0/2578382696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121160 0x55cabf12e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 0x55cabf131a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 -- 172.21.15.39:0/2578382696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 msgr2=0x55cabf131a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 0x55cabf131a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 -- 172.21.15.39:0/2578382696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121160 msgr2=0x55cabf12e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121160 0x55cabf12e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 -- 172.21.15.39:0/2578382696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabf120da0 con 0x55cabf121b50 2024-03-16T10:08:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 0x55cabf131a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a6ea00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02c7b0 tx=0x7f3a5c02cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dfc778b7c28a9e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a58c00640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c04e070 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.322+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f3a5c004450 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a680047f0 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a5c05a050 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 10 client.25358 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cabf19e000 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a680047f0 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3a5c047040 con 0x55cabf121b50 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 10 client.25358 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 10 client.25358 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 20 client.25358 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf1a2b60 0x55cabf1a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 4 client.25358 mds_command: new command op to 24518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.323+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 --> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] -- command(tid 0: {"prefix": "status"}) -- 0x55cabf093190 con 0x55cabf1a2b60 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.324+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf1a2b60 0x55cabf1a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.324+0000 7f3a58c00640 10 client.25358 ms_handle_connect on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.324+0000 7f3a6fe00640 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf1a2b60 0x55cabf1a1870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89172cd7fca7d69a server_cookie=517c2380140e1d2e in_seq=0 out_seq=0 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.324+0000 7f3a58c00640 1 -- 172.21.15.39:0/2578382696 <== mds.4 v2:172.21.15.105:6836/3830303638 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cabf093190 con 0x55cabf1a2b60 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.324+0000 7f3a58c00640 10 client.25358 handle_command_reply: tid=0 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 client.25358 shutdown 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 msgr2=0x55cabf12eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02c7b0 tx=0x7f3a5c02cbe0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 shutdown_connections 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:6836/3830303638,v1:172.21.15.105:6837/3830303638] conn(0x55cabf1a2b60 0x55cabf1a1870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cabf126cc0 0x55cabf131a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cabf121b50 0x55cabf12eef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a58c00640 0 client.25358 ms_handle_reset on v2:172.21.15.105:6836/3830303638 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 --2- 172.21.15.39:0/2578382696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cabf121160 0x55cabf12e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 >> 172.21.15.39:0/2578382696 conn(0x55cabf11c6d0 msgr2=0x55cabf11b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 shutdown_connections 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 1 -- 172.21.15.39:0/2578382696 wait complete. 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.325+0000 7f3a78bfd740 20 client.25358 trim_cache size 0 max 16384 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-03-16T10:08:53.331 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24518, 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 133.76568748899999, 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 188.10209362399999 2024-03-16T10:08:53.332 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 msgr2=0x7f3a540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 0x7f3a540a4cc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a6009b780 tx=0x7f3a60004a40 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 msgr2=0x7f3a70197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000e9f0 tx=0x7f3a6000edf0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 shutdown_connections 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f3a540a2800 0x7f3a540a4cc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a70107560 0x7f3a7019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a70104c30 0x7f3a701980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 --2- 172.21.15.39:0/2824300649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a70102300 0x7f3a70197bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 >> 172.21.15.39:0/2824300649 conn(0x7f3a700fbc70 msgr2=0x7f3a70102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 shutdown_connections 2024-03-16T10:08:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.326+0000 7f3a76200640 1 -- 172.21.15.39:0/2824300649 wait complete. 2024-03-16T10:08:53.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 -- 172.21.15.39:0/417228675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2954104c10 msgr2=0x7f2954105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 --2- 172.21.15.39:0/417228675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2954104c10 0x7f2954105090 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 -- 172.21.15.39:0/417228675 shutdown_connections 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 --2- 172.21.15.39:0/417228675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29541055d0 0x7f295410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 --2- 172.21.15.39:0/417228675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2954104c10 0x7f2954105090 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 --2- 172.21.15.39:0/417228675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954103a20 0x7f2954103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 -- 172.21.15.39:0/417228675 >> 172.21.15.39:0/417228675 conn(0x7f29540fef90 msgr2=0x7f29541013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.553+0000 7f2959a00640 1 -- 172.21.15.39:0/417228675 shutdown_connections 2024-03-16T10:08:53.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.554+0000 7f2959a00640 1 -- 172.21.15.39:0/417228675 wait complete. 2024-03-16T10:08:53.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.555+0000 7f2959a00640 1 Processor -- start 2024-03-16T10:08:53.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.555+0000 7f2959a00640 1 -- start start 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 0x7f2954199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 0x7f295419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29541055d0 0x7f29541a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f295410f6d0 con 0x7f2954103a20 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f295410f550 con 0x7f29541055d0 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2959a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f295410f850 con 0x7f2954104c10 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 0x7f2954199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 0x7f295419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 0x7f2954199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36760/0 (socket says 172.21.15.39:36760) 2024-03-16T10:08:53.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952a00640 1 -- 172.21.15.39:0/4013109793 learned_addr learned my addr 172.21.15.39:0/4013109793 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2953400640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29541055d0 0x7f29541a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 -- 172.21.15.39:0/4013109793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29541055d0 msgr2=0x7f29541a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29541055d0 0x7f29541a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 -- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 msgr2=0x7f2954199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 0x7f2954199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.556+0000 7f2952000640 1 -- 172.21.15.39:0/4013109793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29541a1960 con 0x7f2954104c10 2024-03-16T10:08:53.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f2952000640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 0x7f295419a280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f293c036040 tx=0x7f293c02c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f294b400640 1 -- 172.21.15.39:0/4013109793 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c043020 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f294b400640 1 -- 172.21.15.39:0/4013109793 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c0054e0 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f294b400640 1 -- 172.21.15.39:0/4013109793 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c003c30 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a1bf0 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.557+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a2160 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f294b400640 1 -- 172.21.15.39:0/4013109793 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f293c046020 con 0x7f2954104c10 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 Processor -- start 2024-03-16T10:08:53.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 -- start start 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392987580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x55739298f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5573928ebc50 con 0x557392987160 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5573928949d0 con 0x55739298ccc0 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f295c60d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5573928e7cf0 con 0x557392987b50 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f2952a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f2952000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x55739298f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f2953400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392987580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f2952a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41366/0 (socket says 172.21.15.39:41366) 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.562+0000 7f2952a00640 1 -- 172.21.15.39:0/2947983937 learned_addr learned my addr 172.21.15.39:0/2947983937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 -- 172.21.15.39:0/2947983937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 msgr2=0x55739298f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x55739298f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 -- 172.21.15.39:0/2947983937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 msgr2=0x557392987580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392987580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 -- 172.21.15.39:0/2947983937 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573928aef10 con 0x557392987b50 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2952a00640 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29440115f0 tx=0x7f2944011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c386370433a94b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f294b400640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 0x7f29380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.563+0000 7f2943e00640 1 -- 172.21.15.39:0/2947983937 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944004b20 con 0x557392987b50 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f2943e00640 1 -- 172.21.15.39:0/2947983937 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2944004cc0 con 0x557392987b50 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f2943e00640 1 -- 172.21.15.39:0/2947983937 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944005500 con 0x557392987b50 2024-03-16T10:08:53.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f294b400640 1 -- 172.21.15.39:0/4013109793 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f293c0f0d20 con 0x7f2954104c10 2024-03-16T10:08:53.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f2953400640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 0x7f29380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f295c60d740 1 -- 172.21.15.39:0/2947983937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 msgr2=0x55739298c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.564+0000 7f295c60d740 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29440115f0 tx=0x7f2944011ab0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 -- 172.21.15.39:0/2947983937 shutdown_connections 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x55739298f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298c650 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 --2- 172.21.15.39:0/2947983937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392987580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 -- 172.21.15.39:0/2947983937 >> 172.21.15.39:0/2947983937 conn(0x5573929826d0 msgr2=0x557392982ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f2953400640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 0x7f29380a4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5573929883b0 tx=0x7f294c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 -- 172.21.15.39:0/2947983937 shutdown_connections 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 -- 172.21.15.39:0/2947983937 wait complete. 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.565+0000 7f295c60d740 1 -- start start 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 Processor -- start 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x557392991740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392998780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55739298fc70 con 0x557392987160 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5573928a7e20 con 0x55739298ccc0 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55739298fdf0 con 0x557392987b50 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f295c60d740 10 client.0 _ll_register_callbacks cb 0x5573928a77f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f2952a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f2952a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36788/0 (socket says 172.21.15.39:36788) 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f2952a00640 1 -- 172.21.15.39:0/409097141 learned_addr learned my addr 172.21.15.39:0/409097141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f2952000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x557392991740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.566+0000 7f2953400640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392998780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 -- 172.21.15.39:0/409097141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 msgr2=0x557392998780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392998780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 -- 172.21.15.39:0/409097141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 msgr2=0x557392991200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 -- 172.21.15.39:0/409097141 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573926002d0 con 0x557392987b50 2024-03-16T10:08:53.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952a00640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2952000640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x557392991740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f293c02c7b0 tx=0x7f293c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33409bbce08774b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2940c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b7e60 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f293c004410 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2944004cc0 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.567+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f293c0bce30 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 10 client.25375 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573926ca6b0 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2944004cc0 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f293c0ee4d0 con 0x557392987b50 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 10 client.25375 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 10 client.25375 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 20 client.25375 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x55739299f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 4 client.25375 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:53.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 --> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573929492b0 con 0x55739299e0a0 2024-03-16T10:08:53.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.568+0000 7f2952a00640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x55739299f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.569+0000 7f2952a00640 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x55739299f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ef321929c77b7f5 server_cookie=7c464eaae6960626 in_seq=0 out_seq=0 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.569+0000 7f2940c00640 10 client.25375 ms_handle_connect on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f2940c00640 1 -- 172.21.15.39:0/409097141 <== mds.3 v2:172.21.15.194:6836/3973444513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573929492b0 con 0x55739299e0a0 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f2940c00640 10 client.25375 handle_command_reply: tid=0 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 client.25375 shutdown 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 msgr2=0x557392991740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x557392991740 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f293c02c7b0 tx=0x7f293c0035f0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 shutdown_connections 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x55739299f510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392998780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f2940c00640 0 client.25375 ms_handle_reset on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x557392991740 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 --2- 172.21.15.39:0/409097141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392991200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 >> 172.21.15.39:0/409097141 conn(0x5573929826d0 msgr2=0x557392981a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 shutdown_connections 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 1 -- 172.21.15.39:0/409097141 wait complete. 2024-03-16T10:08:53.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.570+0000 7f295c60d740 20 client.25375 trim_cache size 0 max 16384 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.590+0000 7f295c60d740 1 Processor -- start 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.590+0000 7f295c60d740 1 -- start start 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392990220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55739298ccc0 0x557392981720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55739299d850 con 0x557392987160 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55739299df20 con 0x557392987b50 2024-03-16T10:08:53.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f295c60d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55739299e5f0 con 0x55739298ccc0 2024-03-16T10:08:53.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54658/0 (socket says 172.21.15.39:54658) 2024-03-16T10:08:53.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 -- 172.21.15.39:0/550497402 learned_addr learned my addr 172.21.15.39:0/550497402 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952a00640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392990220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2953400640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55739298ccc0 0x557392981720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 -- 172.21.15.39:0/550497402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55739298ccc0 msgr2=0x557392981720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55739298ccc0 0x557392981720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 -- 172.21.15.39:0/550497402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 msgr2=0x557392990220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.591+0000 7f2952000640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392990220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2952000640 1 -- 172.21.15.39:0/550497402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557392958750 con 0x557392987b50 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2952a00640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392990220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2952000640 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f293c0b9cc0 tx=0x7f293c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12dd01d08bfb1e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2943400640 1 -- 172.21.15.39:0/550497402 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b7e60 con 0x557392987b50 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2943400640 1 -- 172.21.15.39:0/550497402 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c005690 con 0x557392987b50 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.592+0000 7f2943400640 1 -- 172.21.15.39:0/550497402 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0bcc50 con 0x557392987b50 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- 172.21.15.39:0/550497402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 msgr2=0x5573929811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f293c0b9cc0 tx=0x7f293c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:08:53.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- 172.21.15.39:0/550497402 shutdown_connections 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55739298ccc0 0x557392981720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557392987b50 0x5573929811e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 --2- 172.21.15.39:0/550497402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x557392990220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- 172.21.15.39:0/550497402 >> 172.21.15.39:0/550497402 conn(0x5573929826d0 msgr2=0x557392982ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- 172.21.15.39:0/550497402 shutdown_connections 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- 172.21.15.39:0/550497402 wait complete. 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 -- start start 2024-03-16T10:08:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.593+0000 7f295c60d740 1 Processor -- start 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x55739298a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55739299f390 con 0x557392987160 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55739299ecc0 con 0x55739298ccc0 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55739299fa60 con 0x557392987b50 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f295c60d740 10 client.0 _ll_register_callbacks cb 0x7f2944004c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2953400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41406/0 (socket says 172.21.15.39:41406) 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2953400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54668/0 (socket says 172.21.15.39:54668) 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952000640 1 -- 172.21.15.39:0/2206994977 learned_addr learned my addr 172.21.15.39:0/2206994977 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x55739298a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952000640 1 -- 172.21.15.39:0/2206994977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 msgr2=0x557392996ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.594+0000 7f2952000640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2952000640 1 -- 172.21.15.39:0/2206994977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 msgr2=0x55739298a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2952000640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x55739298a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2952000640 1 -- 172.21.15.39:0/2206994977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55739298c9d0 con 0x557392987b50 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2953400640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2952000640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b5da1ef32fa76e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2940c00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b7e60 con 0x557392987b50 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f293c0dd0b0 con 0x557392987b50 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573928f9190 con 0x557392987b50 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f293c0bcc50 con 0x557392987b50 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f295c60d740 10 client.25381 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55739298c9d0 con 0x557392987b50 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.595+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5573928f9190 con 0x557392987b50 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f293c0bc740 con 0x557392987b50 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 10 client.25381 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 10 client.25381 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 20 client.25381 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x557392a06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 4 client.25381 mds_command: new command op to 24523 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 --> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] -- command(tid 0: {"prefix": "status"}) -- 0x5573928f34e0 con 0x55739299e0a0 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.596+0000 7f2952a00640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x557392a06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f2952a00640 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x557392a06900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8cf1479eb5672b6e server_cookie=4062d267ae3d110f in_seq=0 out_seq=0 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f2940c00640 10 client.25381 ms_handle_connect on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f2940c00640 1 -- 172.21.15.39:0/2206994977 <== mds.3 v2:172.21.15.194:6836/3973444513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5573928f34e0 con 0x55739299e0a0 2024-03-16T10:08:53.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f2940c00640 10 client.25381 handle_command_reply: tid=0 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 client.25381 shutdown 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 msgr2=0x55739298ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c005e50 comp rx=0 tx=0).stop 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 shutdown_connections 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:6836/3973444513,v1:172.21.15.194:6837/3973444513] conn(0x55739299e0a0 0x557392a06900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55739298ccc0 0x557392996ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557392987b50 0x55739298ae30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 --2- 172.21.15.39:0/2206994977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557392987160 0x55739298a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 >> 172.21.15.39:0/2206994977 conn(0x5573929826d0 msgr2=0x5573929811e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f2940c00640 0 client.25381 ms_handle_reset on v2:172.21.15.194:6836/3973444513 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 shutdown_connections 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 1 -- 172.21.15.39:0/2206994977 wait complete. 2024-03-16T10:08:53.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.597+0000 7f295c60d740 20 client.25381 trim_cache size 0 max 16384 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24523, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 135.05543521499999, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 191.15372916999999 2024-03-16T10:08:53.604 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:53.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 msgr2=0x7f29380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 0x7f29380a4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5573929883b0 tx=0x7f294c005f90 comp rx=0 tx=0).stop 2024-03-16T10:08:53.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 msgr2=0x7f295419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 0x7f295419a280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f293c036040 tx=0x7f293c02c650 comp rx=0 tx=0).stop 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 shutdown_connections 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f29380a2880 0x7f29380a4d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29541055d0 0x7f29541a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2954104c10 0x7f295419a280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 --2- 172.21.15.39:0/4013109793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2954103a20 0x7f2954199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 >> 172.21.15.39:0/4013109793 conn(0x7f29540fef90 msgr2=0x7f2954107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 shutdown_connections 2024-03-16T10:08:53.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.598+0000 7f2959a00640 1 -- 172.21.15.39:0/4013109793 wait complete. 2024-03-16T10:08:53.615 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:53 smithi039 ceph-mon[31510]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:53.615 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:53 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2659952109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:53.615 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:53 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2255769398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:53.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 -- 172.21.15.39:0/3490882324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 msgr2=0x7f7104106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 --2- 172.21.15.39:0/3490882324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f7104106e60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009900 tx=0x7f70f002bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 -- 172.21.15.39:0/3490882324 shutdown_connections 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 --2- 172.21.15.39:0/3490882324 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71041073a0 0x7f71041097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 --2- 172.21.15.39:0/3490882324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f7104106e60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 --2- 172.21.15.39:0/3490882324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7104102140 0x7f7104104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 -- 172.21.15.39:0/3490882324 >> 172.21.15.39:0/3490882324 conn(0x7f71040fbb50 msgr2=0x7f71040fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 -- 172.21.15.39:0/3490882324 shutdown_connections 2024-03-16T10:08:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.830+0000 7f7108a00640 1 -- 172.21.15.39:0/3490882324 wait complete. 2024-03-16T10:08:53.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 Processor -- start 2024-03-16T10:08:53.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 -- start start 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f71041000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71041073a0 0x7f7104100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f710410d390 con 0x7f71041073a0 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f710410d210 con 0x7f7104104a70 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.832+0000 7f7108a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f710410d510 con 0x7f7104102140 2024-03-16T10:08:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41430/0 (socket says 172.21.15.39:41430) 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7102a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f71041000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 -- 172.21.15.39:0/174550101 learned_addr learned my addr 172.21.15.39:0/174550101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103e00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71041073a0 0x7f7104100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 -- 172.21.15.39:0/174550101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 msgr2=0x7f71041000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f71041000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 -- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71041073a0 msgr2=0x7f7104100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71041073a0 0x7f7104100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 -- 172.21.15.39:0/174550101 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7104100ec0 con 0x7f7104102140 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.833+0000 7f7103400640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70f400e9f0 tx=0x7f70f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.834+0000 7f70fbe00640 1 -- 172.21.15.39:0/174550101 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f400cd80 con 0x7f7104102140 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.834+0000 7f70fbe00640 1 -- 172.21.15.39:0/174550101 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f4010910 con 0x7f7104102140 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.834+0000 7f70fbe00640 1 -- 172.21.15.39:0/174550101 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4018990 con 0x7f7104102140 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.834+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71041a1b00 con 0x7f7104102140 2024-03-16T10:08:53.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.834+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a1fd0 con 0x7f7104102140 2024-03-16T10:08:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.837+0000 7f70fbe00640 1 -- 172.21.15.39:0/174550101 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f70f4010ac0 con 0x7f7104102140 2024-03-16T10:08:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.837+0000 7f710cdaa740 1 Processor -- start 2024-03-16T10:08:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.837+0000 7f710cdaa740 1 -- start start 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8d30 0x557902efd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f00330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557902e5cc50 con 0x557902ef8d30 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557902e059d0 con 0x557902efdea0 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f710cdaa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557902e58cf0 con 0x557902ef8340 2024-03-16T10:08:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8d30 0x557902efd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7102a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f00330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41438/0 (socket says 172.21.15.39:41438) 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 -- 172.21.15.39:0/571476880 learned_addr learned my addr 172.21.15.39:0/571476880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 -- 172.21.15.39:0/571476880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 msgr2=0x557902f00330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f00330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 -- 172.21.15.39:0/571476880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8d30 msgr2=0x557902efd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8d30 0x557902efd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7103e00640 1 -- 172.21.15.39:0/571476880 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557902e1ff10 con 0x557902ef8340 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.838+0000 7f7102a00640 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f00330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f7103e00640 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009fc0 tx=0x7f70fc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843da0aac30a03f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f70fbe00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 0x7f70e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f70efe00640 1 -- 172.21.15.39:0/571476880 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc003b70 con 0x557902ef8340 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f70fbe00640 1 -- 172.21.15.39:0/174550101 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f70f40cf1e0 con 0x7f7104102140 2024-03-16T10:08:53.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f70efe00640 1 -- 172.21.15.39:0/571476880 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc003d10 con 0x557902ef8340 2024-03-16T10:08:53.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f70efe00640 1 -- 172.21.15.39:0/571476880 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011aa0 con 0x557902ef8340 2024-03-16T10:08:53.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.839+0000 7f7103400640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 0x7f70e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 -- 172.21.15.39:0/571476880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 msgr2=0x557902ef8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009fc0 tx=0x7f70fc010860 comp rx=0 tx=0).stop 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 -- 172.21.15.39:0/571476880 shutdown_connections 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f00330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f7103400640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 0x7f70e80a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4097bd0 tx=0x7f70f40d2c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8d30 0x557902efd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 --2- 172.21.15.39:0/571476880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8340 0x557902ef8760 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 -- 172.21.15.39:0/571476880 >> 172.21.15.39:0/571476880 conn(0x557902ef38b0 msgr2=0x557902ef3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 -- 172.21.15.39:0/571476880 shutdown_connections 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.840+0000 7f710cdaa740 1 -- 172.21.15.39:0/571476880 wait complete. 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 -- start start 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 Processor -- start 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557902f00e50 con 0x557902efdea0 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557902e18e20 con 0x557902ef8340 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557902f00fd0 con 0x557902ef8d30 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f710cdaa740 10 client.0 _ll_register_callbacks cb 0x557902e187f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.841+0000 7f7103e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7103e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54706/0 (socket says 172.21.15.39:54706) 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7103e00640 1 -- 172.21.15.39:0/1610988741 learned_addr learned my addr 172.21.15.39:0/1610988741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7103e00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 -- 172.21.15.39:0/1610988741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 msgr2=0x557902f02490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 -- 172.21.15.39:0/1610988741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 msgr2=0x557902f09a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f09a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 -- 172.21.15.39:0/1610988741 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557902b712d0 con 0x557902ef8d30 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7103e00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f7102a00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f029d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009230 tx=0x7f70f0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51e955761466d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.842+0000 7f70ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0005d50 con 0x557902ef8d30 2024-03-16T10:08:53.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f70f00046c0 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc003d10 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70f00048c0 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f710cdaa740 10 client.25393 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557902c3b6b0 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc003d10 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70f0003680 con 0x557902ef8d30 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f710cdaa740 10 client.25393 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.843+0000 7f710cdaa740 10 client.25393 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f710cdaa740 20 client.25393 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f0f240 0x557902f106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f710cdaa740 4 client.25393 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 --> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557902eba2b0 con 0x557902f0f240 2024-03-16T10:08:53.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f7103e00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f0f240 0x557902f106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f7103e00640 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f0f240 0x557902f106b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ba9a70f6f87e911 server_cookie=3dcc0bda470cff59 in_seq=0 out_seq=0 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.844+0000 7f70ecc00640 10 client.25393 ms_handle_connect on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.845+0000 7f70ecc00640 1 -- 172.21.15.39:0/1610988741 <== mds.2 v2:172.21.15.194:6838/985236448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557902eba2b0 con 0x557902f0f240 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.845+0000 7f70ecc00640 10 client.25393 handle_command_reply: tid=0 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.845+0000 7f710cdaa740 1 client.25393 shutdown 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 msgr2=0x557902f029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f029d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009230 tx=0x7f70f0004270 comp rx=0 tx=0).stop 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 shutdown_connections 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f0f240 0x557902f106b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f09a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f029d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1610988741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902f02490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 >> 172.21.15.39:0/1610988741 conn(0x557902ef38b0 msgr2=0x557902ef2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f70ecc00640 0 client.25393 ms_handle_reset on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 shutdown_connections 2024-03-16T10:08:53.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 1 -- 172.21.15.39:0/1610988741 wait complete. 2024-03-16T10:08:53.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.846+0000 7f710cdaa740 20 client.25393 trim_cache size 0 max 16384 2024-03-16T10:08:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:53 smithi105 ceph-mon[36559]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:53 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2659952109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:53 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2255769398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:53.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 Processor -- start 2024-03-16T10:08:53.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 -- start start 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902ef3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f00290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557902f0e9f0 con 0x557902efdea0 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557902f0f0c0 con 0x557902ef8340 2024-03-16T10:08:53.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.872+0000 7f710cdaa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557902f0f790 con 0x557902ef8d30 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7102a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902ef3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f00290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7102a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41442/0 (socket says 172.21.15.39:41442) 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7102a00640 1 -- 172.21.15.39:0/2399601290 learned_addr learned my addr 172.21.15.39:0/2399601290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 -- 172.21.15.39:0/2399601290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 msgr2=0x557902f099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f099d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 -- 172.21.15.39:0/2399601290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 msgr2=0x557902ef3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902ef3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 -- 172.21.15.39:0/2399601290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557902ef7f80 con 0x557902efdea0 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.873+0000 7f7103e00640 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f00290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x557902f11830 tx=0x7f70fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9065313e85759739 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.874+0000 7f70ef400640 1 -- 172.21.15.39:0/2399601290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011b50 con 0x557902efdea0 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.874+0000 7f70ef400640 1 -- 172.21.15.39:0/2399601290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc02f0c0 con 0x557902efdea0 2024-03-16T10:08:53.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.874+0000 7f70ef400640 1 -- 172.21.15.39:0/2399601290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc005320 con 0x557902efdea0 2024-03-16T10:08:53.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.874+0000 7f710cdaa740 1 -- 172.21.15.39:0/2399601290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 msgr2=0x557902f00290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f00290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x557902f11830 tx=0x7f70fc005fd0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 -- 172.21.15.39:0/2399601290 shutdown_connections 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902efdea0 0x557902f00290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 --2- 172.21.15.39:0/2399601290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902ef8340 0x557902ef3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 -- 172.21.15.39:0/2399601290 >> 172.21.15.39:0/2399601290 conn(0x557902ef38b0 msgr2=0x557902ef6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 -- 172.21.15.39:0/2399601290 shutdown_connections 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 -- 172.21.15.39:0/2399601290 wait complete. 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 -- start start 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.875+0000 7f710cdaa740 1 Processor -- start 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8340 0x557902f04e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f08340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557902f10530 con 0x557902ef8340 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557902f0fe60 con 0x557902efdea0 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557902f10c00 con 0x557902ef8d30 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f710cdaa740 10 client.0 _ll_register_callbacks cb 0x7f70f002c620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f7102a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f7102a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41456/0 (socket says 172.21.15.39:41456) 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f7103e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8340 0x557902f04e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.876+0000 7f7102a00640 1 -- 172.21.15.39:0/1820306087 learned_addr learned my addr 172.21.15.39:0/1820306087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 -- 172.21.15.39:0/1820306087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 msgr2=0x557902f08340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f08340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 -- 172.21.15.39:0/1820306087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8340 msgr2=0x557902f04e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8340 0x557902f04e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 -- 172.21.15.39:0/1820306087 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557902f08930 con 0x557902ef8d30 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f7102a00640 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70f002c180 tx=0x7f70f0003f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9de6bb32b6b1a4c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f70ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00035f0 con 0x557902ef8d30 2024-03-16T10:08:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f70f0003790 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.877+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc02f0c0 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70f0034440 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557902f09ca0 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70fc02f0c0 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70f0048530 con 0x557902ef8d30 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 10 client.25399 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 20 client.25399 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.878+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f797c0 0x557902f784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.879+0000 7f710cdaa740 4 client.25399 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.879+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 --> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] -- command(tid 0: {"prefix": "status"}) -- 0x557902e6a190 con 0x557902f797c0 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.879+0000 7f7103e00640 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f797c0 0x557902f784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.879+0000 7f70ecc00640 10 client.25399 ms_handle_connect on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.879+0000 7f7103e00640 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f797c0 0x557902f784d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=941d2662bb142517 server_cookie=7287bd1d11043f1b in_seq=0 out_seq=0 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f70ecc00640 1 -- 172.21.15.39:0/1820306087 <== mds.2 v2:172.21.15.194:6838/985236448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557902e6a190 con 0x557902f797c0 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f70ecc00640 10 client.25399 handle_command_reply: tid=0 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 client.25399 shutdown 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 msgr2=0x557902f05370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70f002c180 tx=0x7f70f0003f70 comp rx=0 tx=0).stop 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 shutdown_connections 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:6838/985236448,v1:172.21.15.194:6839/985236448] conn(0x557902f797c0 0x557902f784d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557902efdea0 0x557902f08340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f70ecc00640 0 client.25399 ms_handle_reset on v2:172.21.15.194:6838/985236448 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557902ef8d30 0x557902f05370 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 --2- 172.21.15.39:0/1820306087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557902ef8340 0x557902f04e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 >> 172.21.15.39:0/1820306087 conn(0x557902ef38b0 msgr2=0x557902ef6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 shutdown_connections 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 1 -- 172.21.15.39:0/1820306087 wait complete. 2024-03-16T10:08:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.880+0000 7f710cdaa740 20 client.25399 trim_cache size 0 max 16384 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24544, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 136.35291504899999, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:53.885 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 185.79062261999999 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 msgr2=0x7f70e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 0x7f70e80a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f70f4097bd0 tx=0x7f70f40d2c20 comp rx=0 tx=0).stop 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 msgr2=0x7f7104101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70f400e9f0 tx=0x7f70f400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 shutdown_connections 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f70e80a2880 0x7f70e80a4d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71041073a0 0x7f7104100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7104104a70 0x7f71041000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 --2- 172.21.15.39:0/174550101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7104102140 0x7f7104101ab0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 >> 172.21.15.39:0/174550101 conn(0x7f71040fbb50 msgr2=0x7f7104102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 shutdown_connections 2024-03-16T10:08:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:53.881+0000 7f7108a00640 1 -- 172.21.15.39:0/174550101 wait complete. 2024-03-16T10:08:53.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-16T10:08:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:53 smithi194 ceph-mon[36445]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-03-16T10:08:53.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:53 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2659952109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:08:53.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:53 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2255769398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:08:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.130+0000 7f1d29a00640 1 -- 172.21.15.39:0/3296818630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241028b0 msgr2=0x7f1d24102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.130+0000 7f1d29a00640 1 --2- 172.21.15.39:0/3296818630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241028b0 0x7f1d24102d30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009900 tx=0x7f1d0c02bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 -- 172.21.15.39:0/3296818630 shutdown_connections 2024-03-16T10:08:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 --2- 172.21.15.39:0/3296818630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d24109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 --2- 172.21.15.39:0/3296818630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241028b0 0x7f1d24102d30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 --2- 172.21.15.39:0/3296818630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241016c0 0x7f1d24101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 -- 172.21.15.39:0/3296818630 >> 172.21.15.39:0/3296818630 conn(0x7f1d240fcc50 msgr2=0x7f1d240ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 -- 172.21.15.39:0/3296818630 shutdown_connections 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.131+0000 7f1d29a00640 1 -- 172.21.15.39:0/3296818630 wait complete. 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.132+0000 7f1d29a00640 1 Processor -- start 2024-03-16T10:08:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.132+0000 7f1d29a00640 1 -- start start 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241028b0 0x7f1d2419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d241a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d340 con 0x7f1d241028b0 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d1c0 con 0x7f1d241016c0 2024-03-16T10:08:54.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d4c0 con 0x7f1d24103270 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54752/0 (socket says 172.21.15.39:54752) 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 -- 172.21.15.39:0/2781702475 learned_addr learned my addr 172.21.15.39:0/2781702475 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241028b0 0x7f1d2419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29000640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d241a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 -- 172.21.15.39:0/2781702475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 msgr2=0x7f1d241a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d241a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 -- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241028b0 msgr2=0x7f1d2419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241028b0 0x7f1d2419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d23e00640 1 -- 172.21.15.39:0/2781702475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d241a17c0 con 0x7f1d241016c0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.133+0000 7f1d29000640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d241a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d23e00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400e9c0 tx=0x7f1d1400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d21600640 1 -- 172.21.15.39:0/2781702475 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400cd80 con 0x7f1d241016c0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d21600640 1 -- 172.21.15.39:0/2781702475 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d140035f0 con 0x7f1d241016c0 2024-03-16T10:08:54.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d21600640 1 -- 172.21.15.39:0/2781702475 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14010690 con 0x7f1d241016c0 2024-03-16T10:08:54.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a1ab0 con 0x7f1d241016c0 2024-03-16T10:08:54.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.134+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a1fd0 con 0x7f1d241016c0 2024-03-16T10:08:54.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.138+0000 7f1d2dc18740 1 Processor -- start 2024-03-16T10:08:54.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d21600640 1 -- 172.21.15.39:0/2781702475 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1d140037a0 con 0x7f1d241016c0 2024-03-16T10:08:54.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 -- start start 2024-03-16T10:08:54.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e6b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e6e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5556c5dcac50 con 0x5556c5e66d30 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556c5d739d0 con 0x5556c5e66340 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c5dc6cf0 con 0x5556c5e6bea0 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d29000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d29000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54764/0 (socket says 172.21.15.39:54764) 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e6e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d29000640 1 -- 172.21.15.39:0/764619275 learned_addr learned my addr 172.21.15.39:0/764619275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.139+0000 7f1d23e00640 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e6b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 -- 172.21.15.39:0/764619275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 msgr2=0x5556c5e6e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e6e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 -- 172.21.15.39:0/764619275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 msgr2=0x5556c5e6b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e6b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 -- 172.21.15.39:0/764619275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c5d8df10 con 0x5556c5e66340 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d29000640 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009fd0 tx=0x7f1d1c0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bd26ce4b89eb93 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d19600640 1 -- 172.21.15.39:0/764619275 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c011760 con 0x5556c5e66340 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d19600640 1 -- 172.21.15.39:0/764619275 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c011de0 con 0x5556c5e66340 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d19600640 1 -- 172.21.15.39:0/764619275 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00e3c0 con 0x5556c5e66340 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.140+0000 7f1d21600640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 0x7f1d080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d21600640 1 -- 172.21.15.39:0/2781702475 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f1d140cf350 con 0x7f1d241016c0 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d23e00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 0x7f1d080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 -- 172.21.15.39:0/764619275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 msgr2=0x5556c5e66760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009fd0 tx=0x7f1d1c0108e0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 -- 172.21.15.39:0/764619275 shutdown_connections 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e6e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e6b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/764619275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e66760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 -- 172.21.15.39:0/764619275 >> 172.21.15.39:0/764619275 conn(0x5556c5e618b0 msgr2=0x5556c5e61cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 -- 172.21.15.39:0/764619275 shutdown_connections 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d2dc18740 1 -- 172.21.15.39:0/764619275 wait complete. 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.141+0000 7f1d23e00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 0x7f1d080a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d1409b8b0 tx=0x7f1d14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 -- start start 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 Processor -- start 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e66340 0x5556c5e704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 0x5556c5e77a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e6ee50 con 0x5556c5e6bea0 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556c5d86e20 con 0x5556c5e66d30 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e6efd0 con 0x5556c5e66340 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.142+0000 7f1d2dc18740 10 client.0 _ll_register_callbacks cb 0x5556c5d867f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:54.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d29000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e66340 0x5556c5e704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d29000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 0x5556c5e77a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54770/0 (socket says 172.21.15.39:54770) 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 -- 172.21.15.39:0/2858835835 learned_addr learned my addr 172.21.15.39:0/2858835835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 -- 172.21.15.39:0/2858835835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e66340 msgr2=0x5556c5e704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e66340 0x5556c5e704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 -- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 msgr2=0x5556c5e77a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 0x5556c5e77a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 -- 172.21.15.39:0/2858835835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c5adf2d0 con 0x5556c5e66d30 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d29000640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 0x5556c5e77a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005940 tx=0x7f1d0c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c494505437fd8e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.143+0000 7f1d12000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c005d50 con 0x5556c5e66d30 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f1d0c005230 con 0x5556c5e66d30 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1c011de0 con 0x5556c5e66d30 2024-03-16T10:08:54.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c005430 con 0x5556c5e66d30 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d2dc18740 10 client.25403 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556c5ba96b0 con 0x5556c5e66d30 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c011de0 con 0x5556c5e66d30 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d0c003a00 con 0x5556c5e66d30 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.144+0000 7f1d2dc18740 10 client.25403 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d2dc18740 10 client.25403 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d2dc18740 20 client.25403 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5e7d290 0x5556c5e7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d2dc18740 4 client.25403 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 --> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556c5e282b0 con 0x5556c5e7d290 2024-03-16T10:08:54.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d29000640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5e7d290 0x5556c5e7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d29000640 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5e7d290 0x5556c5e7e700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16708f3c95d5b5a8 server_cookie=e8be7d1641162b3b in_seq=0 out_seq=0 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.145+0000 7f1d12000640 10 client.25403 ms_handle_connect on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.146+0000 7f1d12000640 1 -- 172.21.15.39:0/2858835835 <== mds.1 v2:172.21.15.39:6840/1425412361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5556c5e282b0 con 0x5556c5e7d290 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.146+0000 7f1d12000640 10 client.25403 handle_command_reply: tid=0 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.146+0000 7f1d2dc18740 1 client.25403 shutdown 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.146+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 msgr2=0x5556c5e70a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.146+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005940 tx=0x7f1d0c004270 comp rx=0 tx=0).stop 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 shutdown_connections 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e6bea0 0x5556c5e77a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e70a00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5e7d290 0x5556c5e7e700 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2858835835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e66340 0x5556c5e704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 >> 172.21.15.39:0/2858835835 conn(0x5556c5e618b0 msgr2=0x5556c5e60a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d12000640 0 client.25403 ms_handle_reset on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 shutdown_connections 2024-03-16T10:08:54.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2858835835 wait complete. 2024-03-16T10:08:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.147+0000 7f1d2dc18740 20 client.25403 trim_cache size 0 max 16384 2024-03-16T10:08:54.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.166+0000 7f1d2dc18740 1 Processor -- start 2024-03-16T10:08:54.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 -- start start 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7ca40 con 0x5556c5e66d30 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7d110 con 0x5556c5e66340 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7d7e0 con 0x5556c5e6bea0 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d29000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d29000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54782/0 (socket says 172.21.15.39:54782) 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d29000640 1 -- 172.21.15.39:0/4072108541 learned_addr learned my addr 172.21.15.39:0/4072108541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.167+0000 7f1d29000640 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 -- 172.21.15.39:0/4072108541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 msgr2=0x5556c5e60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e60720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 -- 172.21.15.39:0/4072108541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 msgr2=0x5556c5e601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 -- 172.21.15.39:0/4072108541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556c5e61450 con 0x5556c5e66d30 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d29000640 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d1be00640 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e77a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c003d70 tx=0x7f1d0c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0b3ff886fdfcf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d18c00640 1 -- 172.21.15.39:0/4072108541 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c005d50 con 0x5556c5e66d30 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d18c00640 1 -- 172.21.15.39:0/4072108541 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c0346a0 con 0x5556c5e66d30 2024-03-16T10:08:54.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.168+0000 7f1d18c00640 1 -- 172.21.15.39:0/4072108541 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0536e0 con 0x5556c5e66d30 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- 172.21.15.39:0/4072108541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 msgr2=0x5556c5e77a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e77a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c003d70 tx=0x7f1d0c009620 comp rx=0 tx=0).stop 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- 172.21.15.39:0/4072108541 shutdown_connections 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66d30 0x5556c5e77a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/4072108541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66340 0x5556c5e601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- 172.21.15.39:0/4072108541 >> 172.21.15.39:0/4072108541 conn(0x5556c5e618b0 msgr2=0x5556c5e5c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- 172.21.15.39:0/4072108541 shutdown_connections 2024-03-16T10:08:54.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- 172.21.15.39:0/4072108541 wait complete. 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.169+0000 7f1d2dc18740 1 -- start start 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 Processor -- start 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66340 0x5556c5e72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e76420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7e580 con 0x5556c5e66340 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7deb0 con 0x5556c5e66d30 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5556c5e7ec50 con 0x5556c5e6bea0 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.170+0000 7f1d2dc18740 10 client.0 _ll_register_callbacks cb 0x7f1d0c02cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d29000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66340 0x5556c5e72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54788/0 (socket says 172.21.15.39:54788) 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 -- 172.21.15.39:0/2647347650 learned_addr learned my addr 172.21.15.39:0/2647347650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d29000640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e76420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 -- 172.21.15.39:0/2647347650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 msgr2=0x5556c5e76420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e76420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 -- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66340 msgr2=0x5556c5e72f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66340 0x5556c5e72f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 -- 172.21.15.39:0/2647347650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556c5e76a10 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d1be00640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005b90 tx=0x7f1d0c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc02e2e3d2f30ff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.171+0000 7f1d12000640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c005d50 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f1d0c053680 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556c5e6e7b0 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1d0c053880 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 10 client.25409 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556c5e77d10 con 0x5556c5e66d30 2024-03-16T10:08:54.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5556c5e6e7b0 con 0x5556c5e66d30 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1d0c05b020 con 0x5556c5e66d30 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 10 client.25409 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 10 client.25409 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 20 client.25409 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5ee7320 0x5556c5ee3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.172+0000 7f1d2dc18740 4 client.25409 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.173+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 --> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] -- command(tid 0: {"prefix": "status"}) -- 0x5556c5dd8190 con 0x5556c5ee7320 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.173+0000 7f1d29000640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5ee7320 0x5556c5ee3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.173+0000 7f1d29000640 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5ee7320 0x5556c5ee3e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b358e899d6bccfac server_cookie=f40bdebf9e6e2e52 in_seq=0 out_seq=0 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.173+0000 7f1d12000640 10 client.25409 ms_handle_connect on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d12000640 1 -- 172.21.15.39:0/2647347650 <== mds.1 v2:172.21.15.39:6840/1425412361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5556c5dd8190 con 0x5556c5ee7320 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d12000640 10 client.25409 handle_command_reply: tid=0 2024-03-16T10:08:54.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 client.25409 shutdown 2024-03-16T10:08:54.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 msgr2=0x5556c5e73470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c005b90 tx=0x7f1d0c0038a0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 shutdown_connections 2024-03-16T10:08:54.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:6840/1425412361,v1:172.21.15.39:6841/1425412361] conn(0x5556c5ee7320 0x5556c5ee3e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5556c5e6bea0 0x5556c5e76420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556c5e66d30 0x5556c5e73470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 --2- 172.21.15.39:0/2647347650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5556c5e66340 0x5556c5e72f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d12000640 0 client.25409 ms_handle_reset on v2:172.21.15.39:6840/1425412361 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.174+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 >> 172.21.15.39:0/2647347650 conn(0x5556c5e618b0 msgr2=0x5556c5e601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 shutdown_connections 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d2dc18740 1 -- 172.21.15.39:0/2647347650 wait complete. 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d2dc18740 20 client.25409 trim_cache size 0 max 16384 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:54.180 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24557, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 137.67120634599999, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 177.95862645899999 2024-03-16T10:08:54.181 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:54.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 msgr2=0x7f1d080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 0x7f1d080a4cc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d1409b8b0 tx=0x7f1d14000bc0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 msgr2=0x7f1d24199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.175+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400e9c0 tx=0x7f1d1400ee80 comp rx=0 tx=0).stop 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 shutdown_connections 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f1d080a2800 0x7f1d080a4cc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1d24103270 0x7f1d241a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d241028b0 0x7f1d2419a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 --2- 172.21.15.39:0/2781702475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d241016c0 0x7f1d24199ba0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 >> 172.21.15.39:0/2781702475 conn(0x7f1d240fcc50 msgr2=0x7f1d24073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 shutdown_connections 2024-03-16T10:08:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.176+0000 7f1d29a00640 1 -- 172.21.15.39:0/2781702475 wait complete. 2024-03-16T10:08:54.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 -- 172.21.15.39:0/1296177240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd981038e0 msgr2=0x7fdd98103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 --2- 172.21.15.39:0/1296177240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd981038e0 0x7fdd98103d00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009900 tx=0x7fdd8802bc40 comp rx=0 tx=0).stop 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 -- 172.21.15.39:0/1296177240 shutdown_connections 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 --2- 172.21.15.39:0/1296177240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd98105490 0x7fdd9810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 --2- 172.21.15.39:0/1296177240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98104ad0 0x7fdd98104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 --2- 172.21.15.39:0/1296177240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd981038e0 0x7fdd98103d00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 -- 172.21.15.39:0/1296177240 >> 172.21.15.39:0/1296177240 conn(0x7fdd980fee50 msgr2=0x7fdd98101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 -- 172.21.15.39:0/1296177240 shutdown_connections 2024-03-16T10:08:54.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.401+0000 7fdd97200640 1 -- 172.21.15.39:0/1296177240 wait complete. 2024-03-16T10:08:54.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.403+0000 7fdd97200640 1 Processor -- start 2024-03-16T10:08:54.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 -- start start 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd981038e0 0x7fdd98199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98105490 0x7fdd981a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810f530 con 0x7fdd981038e0 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810f3b0 con 0x7fdd98104ad0 2024-03-16T10:08:54.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd97200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810f6b0 con 0x7fdd98105490 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.404+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd981038e0 0x7fdd98199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54800/0 (socket says 172.21.15.39:54800) 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 -- 172.21.15.39:0/2761225388 learned_addr learned my addr 172.21.15.39:0/2761225388 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd96800640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98105490 0x7fdd981a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 -- 172.21.15.39:0/2761225388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98105490 msgr2=0x7fdd981a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98105490 0x7fdd981a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 -- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd981038e0 msgr2=0x7fdd98199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd981038e0 0x7fdd98199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 -- 172.21.15.39:0/2761225388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd981a17a0 con 0x7fdd98104ad0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.405+0000 7fdd95400640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ea60 tx=0x7fdd8400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.406+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2761225388 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400cde0 con 0x7fdd98104ad0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.406+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2761225388 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd840035f0 con 0x7fdd98104ad0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.406+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd981a1a90 con 0x7fdd98104ad0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.406+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2761225388 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84010670 con 0x7fdd98104ad0 2024-03-16T10:08:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.406+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a1fe0 con 0x7fdd98104ad0 2024-03-16T10:08:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.409+0000 7fdd9fa2c740 1 Processor -- start 2024-03-16T10:08:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.409+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2761225388 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fdd840037a0 con 0x7fdd98104ad0 2024-03-16T10:08:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.409+0000 7fdd9fa2c740 1 -- start start 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492d30 0x55bbe5497830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 0x55bbe549a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbe53f6c50 con 0x55bbe5492d30 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbe539f9d0 con 0x55bbe5497ea0 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bbe53f2cf0 con 0x55bbe5492340 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 0x55bbe549a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd96800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd96800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41530/0 (socket says 172.21.15.39:41530) 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd96800640 1 -- 172.21.15.39:0/1801802479 learned_addr learned my addr 172.21.15.39:0/1801802479 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95e00640 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492d30 0x55bbe5497830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 -- 172.21.15.39:0/1801802479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 msgr2=0x55bbe5492760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 -- 172.21.15.39:0/1801802479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492d30 msgr2=0x55bbe5497830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492d30 0x55bbe5497830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd95400640 1 -- 172.21.15.39:0/1801802479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe53b9f10 con 0x55bbe5497ea0 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.410+0000 7fdd96800640 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd95400640 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 0x55bbe549a330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd840cc7f0 tx=0x7fdd840d75f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111ff0b54b68c82f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd8ea00640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 0x7fdd7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd83000640 1 -- 172.21.15.39:0/1801802479 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840e2070 con 0x55bbe5497ea0 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd83000640 1 -- 172.21.15.39:0/1801802479 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8409bdf0 con 0x55bbe5497ea0 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd83000640 1 -- 172.21.15.39:0/1801802479 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840afca0 con 0x55bbe5497ea0 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2761225388 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fdd840cf160 con 0x7fdd98104ad0 2024-03-16T10:08:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.411+0000 7fdd96800640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 0x7fdd7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1801802479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 msgr2=0x55bbe549a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 0x55bbe549a330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd840cc7f0 tx=0x7fdd840d75f0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1801802479 shutdown_connections 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5497ea0 0x55bbe549a330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492d30 0x55bbe5497830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1801802479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe5492760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1801802479 >> 172.21.15.39:0/1801802479 conn(0x55bbe548d8b0 msgr2=0x55bbe548dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd96800640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 0x7fdd7c0a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55bbe5493590 tx=0x7fdd90007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1801802479 shutdown_connections 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.412+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1801802479 wait complete. 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 -- start start 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 Processor -- start 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbe549ae50 con 0x55bbe5492340 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbe53b2e20 con 0x55bbe5492d30 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.413+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bbe549afd0 con 0x55bbe5497ea0 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd9fa2c740 10 client.0 _ll_register_callbacks cb 0x55bbe53b27f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:41546/0 (socket says 172.21.15.39:41546) 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 -- 172.21.15.39:0/355180804 learned_addr learned my addr 172.21.15.39:0/355180804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 -- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 msgr2=0x55bbe549c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 -- 172.21.15.39:0/355180804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 msgr2=0x55bbe549c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd96800640 1 -- 172.21.15.39:0/355180804 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe510b2d0 con 0x55bbe5497ea0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.414+0000 7fdd95400640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd96800640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd90010460 tx=0x7fdd90004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd415470c31b5d0c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd77400640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002b060 con 0x55bbe5497ea0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8409bdf0 con 0x55bbe5497ea0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fdd90016030 con 0x55bbe5497ea0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd90032050 con 0x55bbe5497ea0 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd9fa2c740 10 client.25429 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbe51d56b0 con 0x55bbe5497ea0 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.415+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8409bdf0 con 0x55bbe5497ea0 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd9001a390 con 0x55bbe5497ea0 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 10 client.25429 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 10 client.25429 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 20 client.25429 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe54a9270 0x55bbe54aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 4 client.25429 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbe54542b0 con 0x55bbe54a9270 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd95e00640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe54a9270 0x55bbe54aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd95e00640 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe54a9270 0x55bbe54aa6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1233a243af824e server_cookie=97c05462c8c974d2 in_seq=0 out_seq=0 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.416+0000 7fdd77400640 10 client.25429 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd77400640 1 -- 172.21.15.39:0/355180804 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bbe54542b0 con 0x55bbe54a9270 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd77400640 10 client.25429 handle_command_reply: tid=0 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 client.25429 shutdown 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 msgr2=0x55bbe54a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd90010460 tx=0x7fdd90004ab0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 shutdown_connections 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe54a9270 0x55bbe54aa6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a3980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/355180804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 >> 172.21.15.39:0/355180804 conn(0x55bbe548d8b0 msgr2=0x55bbe548ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.417+0000 7fdd77400640 0 client.25429 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.418+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 shutdown_connections 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.418+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/355180804 wait complete. 2024-03-16T10:08:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.418+0000 7fdd9fa2c740 20 client.25429 trim_cache size 0 max 16384 2024-03-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 Processor -- start 2024-03-16T10:08:54.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 -- start start 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe548dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 0x55bbe549c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54a8a20 con 0x55bbe5497ea0 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54a90f0 con 0x55bbe5492d30 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54a97c0 con 0x55bbe5492340 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe548dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:54812/0 (socket says 172.21.15.39:54812) 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd95400640 1 -- 172.21.15.39:0/2765653405 learned_addr learned my addr 172.21.15.39:0/2765653405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.438+0000 7fdd96800640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 0x55bbe549c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 -- 172.21.15.39:0/2765653405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 msgr2=0x55bbe548dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe548dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 -- 172.21.15.39:0/2765653405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 msgr2=0x55bbe549c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 0x55bbe549c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 -- 172.21.15.39:0/2765653405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbe548d3f0 con 0x55bbe5492d30 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95e00640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe548dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd96800640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 0x55bbe549c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd95400640 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ea30 tx=0x7fdd84003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e183c1c39c466ac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd82600640 1 -- 172.21.15.39:0/2765653405 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840b4990 con 0x55bbe5492d30 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd82600640 1 -- 172.21.15.39:0/2765653405 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd840d2970 con 0x55bbe5492d30 2024-03-16T10:08:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.439+0000 7fdd82600640 1 -- 172.21.15.39:0/2765653405 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840b2be0 con 0x55bbe5492d30 2024-03-16T10:08:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/2765653405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 msgr2=0x55bbe54a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ea30 tx=0x7fdd84003f30 comp rx=0 tx=0).stop 2024-03-16T10:08:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/2765653405 shutdown_connections 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5497ea0 0x55bbe549c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe54a3940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/2765653405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5492340 0x55bbe548dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/2765653405 >> 172.21.15.39:0/2765653405 conn(0x55bbe548d8b0 msgr2=0x55bbe549c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/2765653405 shutdown_connections 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/2765653405 wait complete. 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.440+0000 7fdd9fa2c740 1 -- start start 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 Processor -- start 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54aa560 con 0x55bbe5492340 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54a9e90 con 0x55bbe5492d30 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bbe54aac30 con 0x55bbe5497ea0 2024-03-16T10:08:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd9fa2c740 10 client.0 _ll_register_callbacks cb 0x7fdd9000fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd95400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd95e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36972/0 (socket says 172.21.15.39:36972) 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd95e00640 1 -- 172.21.15.39:0/1403606224 learned_addr learned my addr 172.21.15.39:0/1403606224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.441+0000 7fdd96800640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 -- 172.21.15.39:0/1403606224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 msgr2=0x55bbe54a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 -- 172.21.15.39:0/1403606224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 msgr2=0x55bbe549f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 -- 172.21.15.39:0/1403606224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbe54a2c30 con 0x55bbe5492d30 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd95400640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549f670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdd8401f000 tx=0x7fdd84005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9646d16fade1f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd77400640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840b2e20 con 0x55bbe5492d30 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mon.2 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fdd840999f0 con 0x55bbe5492d30 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd840d2970 con 0x55bbe5492d30 2024-03-16T10:08:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd840b17d0 con 0x55bbe5492d30 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.442+0000 7fdd9fa2c740 10 client.25427 fetch_fsmap learned FSMap version 28 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbe550f040 con 0x55bbe5492d30 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mon.2 v2:172.21.15.105:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd840d2970 con 0x55bbe5492d30 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mon.2 v2:172.21.15.105:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd84097d60 con 0x55bbe5492d30 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 10 client.25427 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 20 client.25427 populate_metadata read hostname 'smithi039' 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe5513610 0x55bbe5510120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 4 client.25427 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 --> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] -- command(tid 0: {"prefix": "status"}) -- 0x55bbe5404190 con 0x55bbe5513610 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.443+0000 7fdd95e00640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe5513610 0x55bbe5510120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd95e00640 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe5513610 0x55bbe5510120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0397fc9fd066268 server_cookie=7e77bf8fd5a5d998 in_seq=0 out_seq=0 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd77400640 10 client.25427 ms_handle_connect on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd77400640 1 -- 172.21.15.39:0/1403606224 <== mds.0 v2:172.21.15.105:6838/2189492991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bbe5404190 con 0x55bbe5513610 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd77400640 10 client.25427 handle_command_reply: tid=0 2024-03-16T10:08:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 client.25427 shutdown 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 msgr2=0x55bbe549f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549f670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdd8401f000 tx=0x7fdd84005ab0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 shutdown_connections 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:6838/2189492991,v1:172.21.15.105:6839/2189492991] conn(0x55bbe5513610 0x55bbe5510120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bbe5497ea0 0x55bbe54a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bbe5492d30 0x55bbe549f670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 --2- 172.21.15.39:0/1403606224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bbe5492340 0x55bbe549f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd77400640 0 client.25427 ms_handle_reset on v2:172.21.15.105:6838/2189492991 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 >> 172.21.15.39:0/1403606224 conn(0x55bbe548d8b0 msgr2=0x55bbe548c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 shutdown_connections 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.444+0000 7fdd9fa2c740 1 -- 172.21.15.39:0/1403606224 wait complete. 2024-03-16T10:08:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd9fa2c740 20 client.25427 trim_cache size 0 max 16384 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "65a28618-e37b-11ee-95c9-87774f69a715", 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24569, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 170.21466117200001, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 28, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 175.400165319 2024-03-16T10:08:54.451 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 msgr2=0x7fdd7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 0x7fdd7c0a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55bbe5493590 tx=0x7fdd90007520 comp rx=0 tx=0).stop 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 msgr2=0x7fdd9819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ea60 tx=0x7fdd8400ef20 comp rx=0 tx=0).stop 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 shutdown_connections 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fdd7c0a2800 0x7fdd7c0a4cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdd98105490 0x7fdd981a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd98104ad0 0x7fdd9819a0c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 --2- 172.21.15.39:0/2761225388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd981038e0 0x7fdd98199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.445+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 >> 172.21.15.39:0/2761225388 conn(0x7fdd980fee50 msgr2=0x7fdd98107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.446+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 shutdown_connections 2024-03-16T10:08:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:54.446+0000 7fdd97200640 1 -- 172.21.15.39:0/2761225388 wait complete. 2024-03-16T10:08:54.464 INFO:teuthology.run_tasks:Running task kclient... 2024-03-16T10:08:54.477 INFO:tasks.kclient:Mounting kernel clients... 2024-03-16T10:08:54.477 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-16T10:08:54.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:54.477 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-03-16T10:08:54.525 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-16T10:08:54.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:54.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-16T10:08:54.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:54.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-03-16T10:08:54.731 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-03-16T10:08:54.731 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T10:08:54.731 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-03-16T10:08:54.731 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-03-16T10:08:54.732 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T10:08:55.262 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.718 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:55 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:08:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:55 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:55 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:57.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 -- 172.21.15.39:0/2907159995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a80746a0 msgr2=0x7f22a8074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:57.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 --2- 172.21.15.39:0/2907159995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a80746a0 0x7f22a8074a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f229c009990 tx=0x7f229c009ca0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 -- 172.21.15.39:0/2907159995 shutdown_connections 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 --2- 172.21.15.39:0/2907159995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a810eb60 0x7f22a8110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 --2- 172.21.15.39:0/2907159995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a80746a0 0x7f22a8074a80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 --2- 172.21.15.39:0/2907159995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a8074fc0 0x7f22a810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 -- 172.21.15.39:0/2907159995 >> 172.21.15.39:0/2907159995 conn(0x7f22a80fdd70 msgr2=0x7f22a8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 -- 172.21.15.39:0/2907159995 shutdown_connections 2024-03-16T10:08:57.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.638+0000 7f22b0800700 1 -- 172.21.15.39:0/2907159995 wait complete. 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.640+0000 7f22b0800700 1 Processor -- start 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.640+0000 7f22b0800700 1 -- start start 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.640+0000 7f22b0800700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.640+0000 7f22b0800700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 0x7f22a81a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22b0800700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 0x7f22a819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22b0800700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114840 con 0x7f22a80746a0 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22b0800700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81146c0 con 0x7f22a810eb60 2024-03-16T10:08:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22b0800700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81149c0 con 0x7f22a8074fc0 2024-03-16T10:08:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ade00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ade00700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36992/0 (socket says 172.21.15.39:36992) 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ade00700 1 -- 172.21.15.39:0/2969599759 learned_addr learned my addr 172.21.15.39:0/2969599759 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ad400700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 0x7f22a81a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ae800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 0x7f22a819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.641+0000 7f22ad400700 1 -- 172.21.15.39:0/2969599759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 msgr2=0x7f22a819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ad400700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 0x7f22a819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ad400700 1 -- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 msgr2=0x7f22a81a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ad400700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ad400700 1 -- 172.21.15.39:0/2969599759 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a819b260 con 0x7f22a8074fc0 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ade00700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ae800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 0x7f22a819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22ad400700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 0x7f22a81a0e80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f229800e9f0 tx=0x7f229800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229800cbd0 con 0x7f22a8074fc0 2024-03-16T10:08:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f229800cd70 con 0x7f22a8074fc0 2024-03-16T10:08:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298018780 con 0x7f22a8074fc0 2024-03-16T10:08:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.642+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a819b550 con 0x7f22a8074fc0 2024-03-16T10:08:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.643+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a81a7810 con 0x7f22a8074fc0 2024-03-16T10:08:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.644+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a8068f90 con 0x7f22a8074fc0 2024-03-16T10:08:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.648+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f2298010430 con 0x7f22a8074fc0 2024-03-16T10:08:57.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.649+0000 7f22a2a00700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 0x7f22900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:08:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.649+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f22980cf430 con 0x7f22a8074fc0 2024-03-16T10:08:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.650+0000 7f22ade00700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 0x7f22900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:08:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.650+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f229809c0c0 con 0x7f22a8074fc0 2024-03-16T10:08:57.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.651+0000 7f22ade00700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 0x7f22900a4dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:08:57.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.961+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22a819bff0 con 0x7f22a8074fc0 2024-03-16T10:08:57.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.962+0000 7f22a2a00700 1 -- 172.21.15.39:0/2969599759 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f229809b9c0 con 0x7f22a8074fc0 2024-03-16T10:08:57.963 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-16T10:08:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 msgr2=0x7f22900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 0x7f22900a4dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c00b260 comp rx=0 tx=0).stop 2024-03-16T10:08:57.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 msgr2=0x7f22a81a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 0x7f22a81a0e80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f229800e9f0 tx=0x7f229800ed00 comp rx=0 tx=0).stop 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 shutdown_connections 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22900a2910 0x7f22900a4dd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22a810eb60 0x7f22a819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80746a0 0x7f22a81a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 --2- 172.21.15.39:0/2969599759 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22a8074fc0 0x7f22a81a0e80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 >> 172.21.15.39:0/2969599759 conn(0x7f22a80fdd70 msgr2=0x7f22a80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 shutdown_connections 2024-03-16T10:08:57.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:57.966+0000 7f22b0800700 1 -- 172.21.15.39:0/2969599759 wait complete. 2024-03-16T10:08:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:08:58 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2969599759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:08:58 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2969599759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:08:58.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:08:58 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2969599759' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:08:59.229 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:08:59.230 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-03-16T10:08:59.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:59.230 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-03-16T10:08:59.283 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:08:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:08:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:08:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:08:59.283 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41830sec preferred_lft 41830sec 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-03-16T10:08:59.284 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:08:59.285 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:08:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:08:59.285 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-03-16T10:08:59.285 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T10:08:59.285 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:08:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:08:59.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:08:59.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:59.568 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:08:59.642 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-03-16T10:08:59.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:59.646 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-03-16T10:08:59.690 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-03-16T10:08:59.690 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:08:59.690 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T10:08:59.690 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T10:08:59.690 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:08:59.690 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:08:59.691 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:08:59.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:08:59.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:08:59.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:08:59.691 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:08:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:08:59.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:08:59.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:08:59.958 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-03-16T10:09:00.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:09:00.000 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-03-16T10:09:00.086 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:09:00.087 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:09:00.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--tmp-cephfs 2024-03-16T10:09:00.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-03-16T10:09:00.087 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:09:00.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:09:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:00 smithi039 ceph-mon[31510]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:09:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:00 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:00.335 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-03-16T10:09:00.335 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:09:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:00 smithi105 ceph-mon[36559]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:09:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:00 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:09:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:00 smithi194 ceph-mon[36445]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-16T10:09:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:00 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:00.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:00.651 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:09:00.651 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-03-16T10:09:00.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-03-16T10:09:00.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T10:09:00.651 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-03-16T10:09:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:09:00.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:00.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:09:00.874 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /tmp/cephfs 2024-03-16T10:09:00.916 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/tmp/cephfs' 2024-03-16T10:09:00.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:09:00.917 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /tmp/cephfs 2024-03-16T10:09:01.006 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-16T10:09:01.007 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-16T10:09:01.007 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/700e2c66-fa54-4588-bbc8-89eda93b2cfd 2024-03-16T10:09:01.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:09:01.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/700e2c66-fa54-4588-bbc8-89eda93b2cfd /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-03-16T10:09:01.136 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-16T10:09:01.136 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-16T10:09:01.806 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-16T10:09:01.806 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-16T10:09:01.806 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.105:3300,172.21.15.194:3300" 2024-03-16T10:09:01.806 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@65a28618-e37b-11ee-95c9-87774f69a715.cephfs=/volumes/qa/sv_1/700e2c66-fa54-4588-bbc8-89eda93b2cfd 2024-03-16T10:09:01.806 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.39:3300/172.21.15.105:3300/172.21.15.194:3300" will pass to kernel 2024-03-16T10:09:01.815 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T10:09:01.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:09:01.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /tmp/cephfs 2024-03-16T10:09:01.930 DEBUG:teuthology.orchestra.run.smithi039:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T10:09:02.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:09:02.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:02.064 INFO:teuthology.orchestra.run.smithi039.stdout:client25444 2024-03-16T10:09:02.064 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:09:02.064 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:09:02.117 DEBUG:teuthology.orchestra.run.smithi039:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T10:09:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:09:02.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:02.226 INFO:teuthology.orchestra.run.smithi039.stdout:client25444 2024-03-16T10:09:02.227 DEBUG:teuthology.orchestra.run.smithi039:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T10:09:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:09:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:02.345 INFO:teuthology.orchestra.run.smithi039.stdout:client25444 2024-03-16T10:09:02.346 DEBUG:teuthology.orchestra.run.smithi039:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/65a28618-e37b-11ee-95c9-87774f69a715.client25444/status') 2024-03-16T10:09:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:02 smithi105 ceph-mon[36559]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:09:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:02 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:02 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:02.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:02 smithi105 ceph-mon[36559]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:09:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:02 smithi194 ceph-mon[36445]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:09:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:02 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:02 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:02.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:02 smithi194 ceph-mon[36445]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:02.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:02 smithi039 ceph-mon[31510]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-16T10:09:02.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:02 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:02.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:02 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:02.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:02 smithi039 ceph-mon[31510]: from='mgr.14152 172.21.15.39:0/3841937683' entity='mgr.x' 2024-03-16T10:09:02.483 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.25444 (3)172.21.15.39:0/3730458768 2024-03-16T10:09:02.483 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2024-03-16T10:09:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2024-03-16T10:09:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2024-03-16T10:09:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:71 bytes copied, 7.7501e-05 s, 916 kB/s 2024-03-16T10:09:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:09:02.496 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T10:09:02.505 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T10:09:02.505 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:09:02.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-16T10:09:02.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 -- 172.21.15.39:0/1952939911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8102850 msgr2=0x7f22d8102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 --2- 172.21.15.39:0/1952939911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8102850 0x7f22d8102cd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009900 tx=0x7f22c402bc40 comp rx=0 tx=0).stop 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 -- 172.21.15.39:0/1952939911 shutdown_connections 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 --2- 172.21.15.39:0/1952939911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8103210 0x7f22d8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 --2- 172.21.15.39:0/1952939911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8102850 0x7f22d8102cd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 --2- 172.21.15.39:0/1952939911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8101660 0x7f22d8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 -- 172.21.15.39:0/1952939911 >> 172.21.15.39:0/1952939911 conn(0x7f22d80fcbf0 msgr2=0x7f22d80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 -- 172.21.15.39:0/1952939911 shutdown_connections 2024-03-16T10:09:02.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.724+0000 7f22d7200640 1 -- 172.21.15.39:0/1952939911 wait complete. 2024-03-16T10:09:02.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.726+0000 7f22d7200640 1 Processor -- start 2024-03-16T10:09:02.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.726+0000 7f22d7200640 1 -- start start 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 0x7f22d8073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 0x7f22d8074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22d810d330 con 0x7f22d8103210 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22d810d1b0 con 0x7f22d8101660 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f22d810d4b0 con 0x7f22d8102850 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d5400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 0x7f22d8073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d5e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36376/0 (socket says 172.21.15.39:36376) 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d5e00640 1 -- 172.21.15.39:0/311519532 learned_addr learned my addr 172.21.15.39:0/311519532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.727+0000 7f22d6800640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 0x7f22d8074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:02.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 -- 172.21.15.39:0/311519532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 msgr2=0x7f22d8073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 0x7f22d8073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 -- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 msgr2=0x7f22d8074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 0x7f22d8074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 -- 172.21.15.39:0/311519532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d80749a0 con 0x7f22d8101660 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d6800640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 0x7f22d8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5400640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 0x7f22d8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:02.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d5e00640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c800cd80 con 0x7f22d8101660 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c80035f0 con 0x7f22d8101660 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8010690 con 0x7f22d8101660 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d81a1a30 con 0x7f22d8101660 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.728+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81a1f00 con 0x7f22d8101660 2024-03-16T10:09:02.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.730+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d804fd90 con 0x7f22d8101660 2024-03-16T10:09:02.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.731+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f22c80037a0 con 0x7f22d8101660 2024-03-16T10:09:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.732+0000 7f22cea00640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 0x7f22bc0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.733+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f22c80cf2b0 con 0x7f22d8101660 2024-03-16T10:09:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.733+0000 7f22d5400640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 0x7f22bc0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.733+0000 7f22d5400640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 0x7f22bc0a4d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22c402c250 tx=0x7f22c402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:02.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.736+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22c809bf40 con 0x7f22d8101660 2024-03-16T10:09:02.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:02.989+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f22d8108980 con 0x7f22d8101660 2024-03-16T10:09:03.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.084+0000 7f22cea00640 1 -- 172.21.15.39:0/311519532 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f22d8108980 con 0x7f22d8101660 2024-03-16T10:09:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 msgr2=0x7f22bc0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 0x7f22bc0a4d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22c402c250 tx=0x7f22c402cf90 comp rx=0 tx=0).stop 2024-03-16T10:09:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 msgr2=0x7f22d8075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ee80 comp rx=0 tx=0).stop 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 shutdown_connections 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f22bc0a2870 0x7f22bc0a4d30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d8103210 0x7f22d8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d8102850 0x7f22d8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 --2- 172.21.15.39:0/311519532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22d8101660 0x7f22d8075590 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 >> 172.21.15.39:0/311519532 conn(0x7f22d80fcbf0 msgr2=0x7f22d8105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.087+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 shutdown_connections 2024-03-16T10:09:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.088+0000 7f22d7200640 1 -- 172.21.15.39:0/311519532 wait complete. 2024-03-16T10:09:03.113 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-16T10:09:03.123 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T10:09:03.131 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T10:09:03.131 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:09:03.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.344+0000 7f1371800640 1 -- 172.21.15.39:0/682764428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c107310 msgr2=0x7f136c109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.344+0000 7f1371800640 1 --2- 172.21.15.39:0/682764428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c107310 0x7f136c109760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ca0 tx=0x7f136402c100 comp rx=0 tx=0).stop 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 -- 172.21.15.39:0/682764428 shutdown_connections 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 --2- 172.21.15.39:0/682764428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c107310 0x7f136c109760 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 --2- 172.21.15.39:0/682764428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 0x7f136c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 --2- 172.21.15.39:0/682764428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c1020b0 0x7f136c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 -- 172.21.15.39:0/682764428 >> 172.21.15.39:0/682764428 conn(0x7f136c0fbac0 msgr2=0x7f136c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:03.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 -- 172.21.15.39:0/682764428 shutdown_connections 2024-03-16T10:09:03.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.345+0000 7f1371800640 1 -- 172.21.15.39:0/682764428 wait complete. 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 Processor -- start 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 -- start start 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 0x7f136c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 0x7f136c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d2a0 con 0x7f136c1049e0 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d120 con 0x7f136c107310 2024-03-16T10:09:03.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f1371800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d420 con 0x7f136c1020b0 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.346+0000 7f136be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f136be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:43804/0 (socket says 172.21.15.39:43804) 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f136be00640 1 -- 172.21.15.39:0/4106539557 learned_addr learned my addr 172.21.15.39:0/4106539557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 0x7f136c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1363e00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 0x7f136c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 -- 172.21.15.39:0/4106539557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 msgr2=0x7f136c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 -- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 msgr2=0x7f136c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 0x7f136c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 -- 172.21.15.39:0/4106539557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c1a16f0 con 0x7f136c107310 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f136be00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.347+0000 7f1370e00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 0x7f136c1a1080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1364037040 tx=0x7f1364035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.348+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364039040 con 0x7f136c107310 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.348+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1364044070 con 0x7f136c107310 2024-03-16T10:09:03.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.348+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c1a1980 con 0x7f136c107310 2024-03-16T10:09:03.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.348+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364004450 con 0x7f136c107310 2024-03-16T10:09:03.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.348+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c1a1e90 con 0x7f136c107310 2024-03-16T10:09:03.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.349+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c108e00 con 0x7f136c107310 2024-03-16T10:09:03.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.351+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f13640045f0 con 0x7f136c107310 2024-03-16T10:09:03.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.353+0000 7f1369600640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 0x7f13500a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.353+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f13640f1410 con 0x7f136c107310 2024-03-16T10:09:03.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.353+0000 7f136be00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 0x7f13500a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:03.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.354+0000 7f136be00640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 0x7f13500a4d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f135c011550 tx=0x7f135c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.358+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f13640be0a0 con 0x7f136c107310 2024-03-16T10:09:03.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/311519532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:03 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/311519532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:03 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/311519532' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:03 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-16T10:09:03.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:03.660+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f136c0623e0 con 0x7f136c107310 2024-03-16T10:09:04.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.103+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f13640ba010 con 0x7f136c107310 2024-03-16T10:09:04.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.104+0000 7f1369600640 1 -- 172.21.15.39:0/4106539557 <== mon.2 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f13640098b0 con 0x7f136c107310 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 msgr2=0x7f13500a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 0x7f13500a4d30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f135c011550 tx=0x7f135c005f90 comp rx=0 tx=0).stop 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 msgr2=0x7f136c1a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 0x7f136c1a1080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1364037040 tx=0x7f1364035d10 comp rx=0 tx=0).stop 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 shutdown_connections 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f13500a2870 0x7f13500a4d30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f136c107310 0x7f136c1a1080 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f136c1049e0 0x7f136c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 --2- 172.21.15.39:0/4106539557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f136c1020b0 0x7f136c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 >> 172.21.15.39:0/4106539557 conn(0x7f136c0fbac0 msgr2=0x7f136c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.111+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 shutdown_connections 2024-03-16T10:09:04.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.112+0000 7f1371800640 1 -- 172.21.15.39:0/4106539557 wait complete. 2024-03-16T10:09:04.133 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-16T10:09:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 -- 172.21.15.39:0/3812214913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c102a00 msgr2=0x7f715c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 --2- 172.21.15.39:0/3812214913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c102a00 0x7f715c102e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7148009900 tx=0x7f714802bc30 comp rx=0 tx=0).stop 2024-03-16T10:09:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 -- 172.21.15.39:0/3812214913 shutdown_connections 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 --2- 172.21.15.39:0/3812214913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 --2- 172.21.15.39:0/3812214913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c102a00 0x7f715c102e80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 --2- 172.21.15.39:0/3812214913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c101810 0x7f715c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 -- 172.21.15.39:0/3812214913 >> 172.21.15.39:0/3812214913 conn(0x7f715c0fcda0 msgr2=0x7f715c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 -- 172.21.15.39:0/3812214913 shutdown_connections 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.329+0000 7f7163a00640 1 -- 172.21.15.39:0/3812214913 wait complete. 2024-03-16T10:09:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 Processor -- start 2024-03-16T10:09:04.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 -- start start 2024-03-16T10:09:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c102a00 0x7f715c19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f715c10d470 con 0x7f715c102a00 2024-03-16T10:09:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f715c10d2f0 con 0x7f715c1033c0 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.331+0000 7f7163a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f715c10d5f0 con 0x7f715c101810 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:43838/0 (socket says 172.21.15.39:43838) 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 -- 172.21.15.39:0/2223936340 learned_addr learned my addr 172.21.15.39:0/2223936340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f715be00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c102a00 0x7f715c19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 msgr2=0x7f715c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c102a00 msgr2=0x7f715c19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c102a00 0x7f715c19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 -- 172.21.15.39:0/2223936340 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1a1870 con 0x7f715c101810 2024-03-16T10:09:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.332+0000 7f7161000640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9c0 tx=0x7f715000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.333+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715000cd80 con 0x7f715c101810 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.333+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71500035f0 con 0x7f715c101810 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.333+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1a1b60 con 0x7f715c101810 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.333+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150010690 con 0x7f715c101810 2024-03-16T10:09:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.333+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c1a20b0 con 0x7f715c101810 2024-03-16T10:09:04.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.335+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c1a2280 con 0x7f715c101810 2024-03-16T10:09:04.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.336+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f71500037a0 con 0x7f715c101810 2024-03-16T10:09:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.338+0000 7f7159600640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 0x7f71440a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.338+0000 7f715be00640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 msgr2=0x7f71440a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.339+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f71500cf2b0 con 0x7f715c101810 2024-03-16T10:09:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.339+0000 7f715be00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 0x7f71440a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:09:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.341+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f715009bf80 con 0x7f715c101810 2024-03-16T10:09:04.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: ignoring --setuser ceph since I am not root 2024-03-16T10:09:04.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: ignoring --setgroup ceph since I am not root 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mgr[37956]: -- 172.21.15.105:0/2745944273 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9ddda85a0 con 0x55d9ddd7d000 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mon[36559]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/4106539557' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mon[36445]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/4106539557' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: ignoring --setuser ceph since I am not root 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: ignoring --setgroup ceph since I am not root 2024-03-16T10:09:04.456 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-mgr[38464]: -- 172.21.15.194:0/2521572986 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c171425a0 con 0x556c17116800 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-mon[31510]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/4106539557' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setuser ceph since I am not root 2024-03-16T10:09:04.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: ignoring --setgroup ceph since I am not root 2024-03-16T10:09:04.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.540+0000 7f715be00640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 msgr2=0x7f71440a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:04.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.540+0000 7f715be00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 0x7f71440a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:09:04.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.601+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f715c1a2450 con 0x7f715c101810 2024-03-16T10:09:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.608+0000 7f7159600640 1 -- 172.21.15.39:0/2223936340 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f715009b880 con 0x7f715c101810 2024-03-16T10:09:04.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.611+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 msgr2=0x7f71440a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:09:04.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.611+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 0x7f71440a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.611+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 msgr2=0x7f715c199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.611+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f715000e9c0 tx=0x7f715000ee80 comp rx=0 tx=0).stop 2024-03-16T10:09:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 shutdown_connections 2024-03-16T10:09:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f71440a28f0 0x7f71440a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f715c1033c0 0x7f715c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f715c102a00 0x7f715c19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 --2- 172.21.15.39:0/2223936340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f715c101810 0x7f715c199c60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 >> 172.21.15.39:0/2223936340 conn(0x7f715c0fcda0 msgr2=0x7f715c073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:04.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 shutdown_connections 2024-03-16T10:09:04.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.612+0000 7f7163a00640 1 -- 172.21.15.39:0/2223936340 wait complete. 2024-03-16T10:09:04.640 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-16T10:09:04.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 -- 172.21.15.39:0/1776226859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 msgr2=0x7f4f3810bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/1776226859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f3810bb80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009860 tx=0x7f4f3002bb70 comp rx=0 tx=0).stop 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 -- 172.21.15.39:0/1776226859 shutdown_connections 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/1776226859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f3810bb80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/1776226859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 0x7f4f380746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/1776226859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f38073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 -- 172.21.15.39:0/1776226859 >> 172.21.15.39:0/1776226859 conn(0x7f4f380fcda0 msgr2=0x7f4f380ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 -- 172.21.15.39:0/1776226859 shutdown_connections 2024-03-16T10:09:04.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.820+0000 7f4f3cc00640 1 -- 172.21.15.39:0/1776226859 wait complete. 2024-03-16T10:09:04.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 Processor -- start 2024-03-16T10:09:04.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 -- start start 2024-03-16T10:09:04.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 0x7f4f38199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f381a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810f690 con 0x7f4f38074240 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810f510 con 0x7f4f38075820 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.821+0000 7f4f3cc00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3810f810 con 0x7f4f381096f0 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36430/0 (socket says 172.21.15.39:36430) 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 -- 172.21.15.39:0/709875491 learned_addr learned my addr 172.21.15.39:0/709875491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f35e00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 0x7f4f38199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f36800640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f381a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 msgr2=0x7f4f381a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f381a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 msgr2=0x7f4f38199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 0x7f4f38199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 -- 172.21.15.39:0/709875491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f381a1850 con 0x7f4f38075820 2024-03-16T10:09:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.822+0000 7f4f2d400640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000ea60 tx=0x7f4f2000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.823+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000cde0 con 0x7f4f38075820 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.823+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f200035f0 con 0x7f4f38075820 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.823+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20010670 con 0x7f4f38075820 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.823+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381a1b40 con 0x7f4f38075820 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.823+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a2040 con 0x7f4f38075820 2024-03-16T10:09:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.824+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f38108420 con 0x7f4f38075820 2024-03-16T10:09:04.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.826+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7f4f200037a0 con 0x7f4f38075820 2024-03-16T10:09:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.827+0000 7f4f2f400640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 0x7f4f1c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.827+0000 7f4f35e00640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 msgr2=0x7f4f1c0a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.827+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7f4f200cf1d0 con 0x7f4f38075820 2024-03-16T10:09:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.828+0000 7f4f35e00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 0x7f4f1c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:09:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:04.831+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4f2009be60 con 0x7f4f38075820 2024-03-16T10:09:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.029+0000 7f4f35e00640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 msgr2=0x7f4f1c0a4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.029+0000 7f4f35e00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 0x7f4f1c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:09:05.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.032+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4f3802d1b0 con 0x7f4f38075820 2024-03-16T10:09:05.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.042+0000 7f4f2f400640 1 -- 172.21.15.39:0/709875491 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f4f2009b760 con 0x7f4f38075820 2024-03-16T10:09:05.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 msgr2=0x7f4f1c0a4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:09:05.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 0x7f4f1c0a4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 msgr2=0x7f4f3819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000ea60 tx=0x7f4f2000ef20 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 shutdown_connections 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7f4f1c0a2970 0x7f4f1c0a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f381096f0 0x7f4f381a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f38075820 0x7f4f3819a180 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 --2- 172.21.15.39:0/709875491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f38074240 0x7f4f38199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 >> 172.21.15.39:0/709875491 conn(0x7f4f380fcda0 msgr2=0x7f4f381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 shutdown_connections 2024-03-16T10:09:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.045+0000 7f4f3cc00640 1 -- 172.21.15.39:0/709875491 wait complete. 2024-03-16T10:09:05.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-16T10:09:05.114 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:04.725+0000 7efd645a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T10:09:05.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:04.844+0000 7efd645a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T10:09:05.115 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:04.804+0000 7f9d58ac4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T10:09:05.115 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:04.899+0000 7f9d58ac4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:04.989+0000 7f720b4d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:05.148+0000 7f720b4d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:05 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:05 smithi039 ceph-mon[31510]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:05 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:05 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 -- 172.21.15.39:0/893085405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501022b0 msgr2=0x7fc150108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 --2- 172.21.15.39:0/893085405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501022b0 0x7fc150108b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc148009ca0 tx=0x7fc14802c0f0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 -- 172.21.15.39:0/893085405 shutdown_connections 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 --2- 172.21.15.39:0/893085405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501022b0 0x7fc150108b40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 --2- 172.21.15.39:0/893085405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1501018f0 0x7fc150101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 --2- 172.21.15.39:0/893085405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc150100700 0x7fc150100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 -- 172.21.15.39:0/893085405 >> 172.21.15.39:0/893085405 conn(0x7fc1500fbc70 msgr2=0x7fc1500fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:05.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 -- 172.21.15.39:0/893085405 shutdown_connections 2024-03-16T10:09:05.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.264+0000 7fc154e00640 1 -- 172.21.15.39:0/893085405 wait complete. 2024-03-16T10:09:05.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.265+0000 7fc154e00640 1 Processor -- start 2024-03-16T10:09:05.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.265+0000 7fc154e00640 1 -- start start 2024-03-16T10:09:05.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc150100700 0x7fc150199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:05.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 0x7fc15019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1501022b0 0x7fc1501a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010c360 con 0x7fc1501022b0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010c1e0 con 0x7fc150100700 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc154e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc15010c4e0 con 0x7fc1501018f0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 0x7fc15019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14de00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc150100700 0x7fc150199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 0x7fc15019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:43874/0 (socket says 172.21.15.39:43874) 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 -- 172.21.15.39:0/4039520240 learned_addr learned my addr 172.21.15.39:0/4039520240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14e800640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1501022b0 0x7fc1501a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc150100700 msgr2=0x7fc150199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc150100700 0x7fc150199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1501022b0 msgr2=0x7fc1501a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1501022b0 0x7fc1501a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14d400640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1501a1830 con 0x7fc1501018f0 2024-03-16T10:09:05.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.266+0000 7fc14e800640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1501022b0 0x7fc1501a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc14d400640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 0x7fc15019a160 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ea60 tx=0x7fc13c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00cde0 con 0x7fc1501018f0 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13c0035f0 con 0x7fc1501018f0 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1501a1b20 con 0x7fc1501018f0 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c010670 con 0x7fc1501018f0 2024-03-16T10:09:05.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.267+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501a2070 con 0x7fc1501018f0 2024-03-16T10:09:05.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.269+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1501a22a0 con 0x7fc1501018f0 2024-03-16T10:09:05.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.273+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137307+0+0 (secure 0 0 0) 0x7fc13c0037a0 con 0x7fc1501018f0 2024-03-16T10:09:05.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.274+0000 7fc146a00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:05.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.275+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:05.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.275+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9216+0+0 (secure 0 0 0) 0x7fc13c0cf300 con 0x7fc1501018f0 2024-03-16T10:09:05.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.275+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:09:05.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.283+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc13c09bf90 con 0x7fc1501018f0 2024-03-16T10:09:05.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T10:09:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-mon[36445]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-16T10:09:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.476+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:05.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.476+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:09:05.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.534+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc1501a2470 con 0x7fc1340a28f0 2024-03-16T10:09:05.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-16T10:09:05.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-mon[36559]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-16T10:09:05.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.573 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' 2024-03-16T10:09:05.831 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:05.573+0000 7efd645a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T10:09:05.832 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:05.686+0000 7efd645a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T10:09:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.877+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:05.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:05.877+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T10:09:05.908 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:05.556+0000 7f9d58ac4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T10:09:05.908 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:05.650+0000 7f9d58ac4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T10:09:05.908 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:05.793+0000 7f9d58ac4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T10:09:05.908 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:05.907+0000 7f9d58ac4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T10:09:06.114 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:05.831+0000 7efd645a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T10:09:06.114 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:05.929+0000 7efd645a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T10:09:06.115 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.026+0000 7efd645a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T10:09:06.205 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:05.991+0000 7f9d58ac4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T10:09:06.391 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.053+0000 7f720b4d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T10:09:06.391 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.201+0000 7f720b4d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T10:09:06.391 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.389+0000 7f720b4d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T10:09:06.553 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:06.259+0000 7f9d58ac4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T10:09:06.553 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:06.370+0000 7f9d58ac4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:06.553 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:06.456+0000 7f9d58ac4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T10:09:06.553 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:06.552+0000 7f9d58ac4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T10:09:06.610 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.294+0000 7efd645a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T10:09:06.610 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.406+0000 7efd645a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:06.610 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.493+0000 7efd645a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T10:09:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:06.678+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:06.678+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T10:09:06.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.519+0000 7f720b4d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T10:09:06.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.635+0000 7f720b4d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T10:09:06.864 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.610+0000 7efd645a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T10:09:06.864 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:06.787+0000 7efd645a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T10:09:06.955 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:06.751+0000 7f9d58ac4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T10:09:07.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:06.985+0000 7f720b4d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T10:09:07.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:07.132+0000 7f720b4d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:07.656 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:07.258+0000 7f720b4d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T10:09:07.656 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:07.403+0000 7f720b4d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T10:09:07.656 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:07.653+0000 7f720b4d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T10:09:08.206 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:07.790+0000 7f9d58ac4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T10:09:08.206 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:07.933+0000 7f9d58ac4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T10:09:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:08.280+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:08.280+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T10:09:08.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:07 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:07.919+0000 7efd645a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T10:09:08.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:08.053+0000 7efd645a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T10:09:08.687 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:08.426+0000 7efd645a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T10:09:08.687 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:08.548+0000 7efd645a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T10:09:08.695 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:08.311+0000 7f9d58ac4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T10:09:08.695 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:08.418+0000 7f9d58ac4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T10:09:08.695 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:08.693+0000 7f9d58ac4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:08.955 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:08.889+0000 7f9d58ac4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T10:09:09.023 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:08.757+0000 7efd645a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:09.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:09.022+0000 7efd645a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T10:09:09.450 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:09.183+0000 7f9d58ac4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:09.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:09.005+0000 7f720b4d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T10:09:09.505 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:09.188+0000 7f720b4d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T10:09:09.682 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:09.379+0000 7efd645a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:09.705 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:09.449+0000 7f9d58ac4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:09.706 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:09.690+0000 7f9d58ac4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T10:09:10.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:09.644+0000 7f720b4d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T10:09:10.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:09.808+0000 7f720b4d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T10:09:10.054 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:09.682+0000 7efd645a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:10.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:10.054+0000 7efd645a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T10:09:10.450 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:10.156+0000 7f720b4d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:10.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:10.448+0000 7f720b4d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T10:09:10.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:10.572+0000 7f9d58ac4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T10:09:11.178 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:10.845+0000 7f720b4d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:09:11.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:11.041+0000 7efd645a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T10:09:11.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:11.484+0000 7fc14de00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/1082942412 2024-03-16T10:09:11.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:11.484+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-16T10:09:11.501 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:11.176+0000 7f720b4d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T10:09:11.527 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:11.211+0000 7f9d58ac4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T10:09:11.527 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:11.310+0000 7f9d58ac4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T10:09:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:11.608+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 20) ==== 137307+0+0 (secure 0 0 0) 0x7fc13c097e70 con 0x7fc1501018f0 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:11.499+0000 7f720b4d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: Standby manager daemon z restarted 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: Standby manager daemon z started 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T10:09:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:11 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:11.864 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:11.700+0000 7efd645a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T10:09:11.864 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:11.800+0000 7efd645a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: Standby manager daemon z restarted 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: Standby manager daemon z started 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T10:09:11.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:11 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z[38460]: 2024-03-16T10:09:11.526+0000 7f9d58ac4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: Standby manager daemon z restarted 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: Standby manager daemon z started 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T10:09:11.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:11 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.194:0/554063767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:12.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y[37952]: 2024-03-16T10:09:12.046+0000 7efd645a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T10:09:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:12.639+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 8 ==== mgrmap(e 21) ==== 137307+0+0 (secure 0 0 0) 0x7fc13c0cca70 con 0x7fc1501018f0 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: Standby manager daemon y restarted 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: Standby manager daemon y started 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T10:09:12.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:12 smithi194 ceph-mon[36445]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:12.637+0000 7f720b4d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: Standby manager daemon y restarted 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: Standby manager daemon y started 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T10:09:13.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:12 smithi039 ceph-mon[31510]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:13.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: Standby manager daemon y restarted 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: Standby manager daemon y started 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T10:09:13.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:12 smithi105 ceph-mon[36559]: from='mgr.? 172.21.15.105:0/3023198461' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:09:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:13 smithi039 ceph-mon[31510]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-16T10:09:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:13.404+0000 7f720b4d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T10:09:13.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:13.544+0000 7f720b4d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T10:09:13.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:13.877+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 9 ==== mgrmap(e 22) ==== 136975+0+0 (secure 0 0 0) 0x7fc13c09baf0 con 0x7fc1501018f0 2024-03-16T10:09:13.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:13.878+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 msgr2=0x7fc1340a4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:09:13.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:13.878+0000 7fc146a00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/1082942412,v1:172.21.15.39:6801/1082942412] conn(0x7fc1340a28f0 0x7fc1340a4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:13.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:13 smithi194 ceph-mon[36445]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-16T10:09:14.083 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:13 smithi105 ceph-mon[36559]: mgrmap e21: x(active, since 11m), standbys: y, z 2024-03-16T10:09:14.093 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:13.851+0000 7f720b4d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T10:09:14.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.245+0000 7f715fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.245+0000 7f715fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.245+0000 7f715fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.245+0000 7f715fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.245+0000 7f715fc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.249+0000 7f715de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.249+0000 7f715de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.249+0000 7f715de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.249+0000 7f715de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:09:14.249+0000 7f715de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:09:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.884+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mon.1 v2:172.21.15.194:3300/0 10 ==== mgrmap(e 23) ==== 137102+0+0 (secure 0 0 0) 0x7fc13c098740 con 0x7fc1501018f0 2024-03-16T10:09:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.884+0000 7fc146a00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc134068250 0x7fc134068630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.884+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc1501a2470 con 0x7fc134068250 2024-03-16T10:09:14.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.894+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc134068250 0x7fc134068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:14.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.896+0000 7fc14de00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc134068250 0x7fc134068630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc15019b0f0 tx=0x7fc140005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:14.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: Active manager daemon x restarted 2024-03-16T10:09:14.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: Activating manager daemon x 2024-03-16T10:09:14.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: mgrmap e22: x(active, starting, since 0.0210284s), standbys: y, z 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:09:14.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:09:14.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: Manager daemon x is now available 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:14.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:14 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:14.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: Active manager daemon x restarted 2024-03-16T10:09:14.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: Activating manager daemon x 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: mgrmap e22: x(active, starting, since 0.0210284s), standbys: y, z 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:09:14.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:09:14.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: Manager daemon x is now available 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:14.959 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:14 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:14.993 INFO:teuthology.orchestra.run.smithi039.stdout:Schedule set for path / 2024-03-16T10:09:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.992+0000 7fc146a00640 1 -- 172.21.15.39:0/4039520240 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc1501a2470 con 0x7fc134068250 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc134068250 msgr2=0x7fc134068630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc134068250 0x7fc134068630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc15019b0f0 tx=0x7fc140005f90 comp rx=0 tx=0).stop 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 msgr2=0x7fc15019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 --2- 172.21.15.39:0/4039520240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1501018f0 0x7fc15019a160 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ea60 tx=0x7fc13c00ef20 comp rx=0 tx=0).stop 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc14e800640 1 -- 172.21.15.39:0/4039520240 reap_dead start 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 shutdown_connections 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.994+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 >> 172.21.15.39:0/4039520240 conn(0x7fc1500fbc70 msgr2=0x7fc1500fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.995+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 shutdown_connections 2024-03-16T10:09:14.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:14.995+0000 7fc154e00640 1 -- 172.21.15.39:0/4039520240 wait complete. 2024-03-16T10:09:15.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: Active manager daemon x restarted 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: Activating manager daemon x 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: osdmap e95: 12 total, 12 up, 12 in 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: mgrmap e22: x(active, starting, since 0.0210284s), standbys: y, z 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T10:09:15.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T10:09:15.116 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:09:15.117 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: Manager daemon x is now available 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:15.118 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:14 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T10:09:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 -- 172.21.15.39:0/708570855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 msgr2=0x7fe75c0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 --2- 172.21.15.39:0/708570855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c0746c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe748009930 tx=0x7fe74802bc20 comp rx=0 tx=0).stop 2024-03-16T10:09:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 -- 172.21.15.39:0/708570855 shutdown_connections 2024-03-16T10:09:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 --2- 172.21.15.39:0/708570855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c10dc20 0x7fe75c110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 --2- 172.21.15.39:0/708570855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c0746c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.192+0000 7fe75b200640 1 --2- 172.21.15.39:0/708570855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c075820 0x7fe75c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.193+0000 7fe75b200640 1 -- 172.21.15.39:0/708570855 >> 172.21.15.39:0/708570855 conn(0x7fe75c0fcbc0 msgr2=0x7fe75c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.193+0000 7fe75b200640 1 -- 172.21.15.39:0/708570855 shutdown_connections 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.193+0000 7fe75b200640 1 -- 172.21.15.39:0/708570855 wait complete. 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.194+0000 7fe75b200640 1 Processor -- start 2024-03-16T10:09:15.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.194+0000 7fe75b200640 1 -- start start 2024-03-16T10:09:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 0x7fe75c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10dc20 0x7fe75c1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1138a0 con 0x7fe75c10dc20 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c113720 con 0x7fe75c074240 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c113a20 con 0x7fe75c075820 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:38920/0 (socket says 172.21.15.39:38920) 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 0x7fe75c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 -- 172.21.15.39:0/837169167 learned_addr learned my addr 172.21.15.39:0/837169167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe75a800640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10dc20 0x7fe75c1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 -- 172.21.15.39:0/837169167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 msgr2=0x7fe75c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 0x7fe75c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 -- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10dc20 msgr2=0x7fe75c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10dc20 0x7fe75c1a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.195+0000 7fe759e00640 1 -- 172.21.15.39:0/837169167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c1a5b90 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe759400640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 0x7fe75c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe759e00640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00e9f0 tx=0x7fe74c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00cd80 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe74c010910 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c018990 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1a5e80 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.196+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a6290 con 0x7fe75c074240 2024-03-16T10:09:15.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.198+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c103df0 con 0x7fe75c074240 2024-03-16T10:09:15.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.199+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 23) ==== 137102+0+0 (secure 0 0 0) 0x7fe74c010ac0 con 0x7fe75c074240 2024-03-16T10:09:15.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.201+0000 7fe752a00640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 0x7fe7400a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.202+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9474+0+0 (secure 0 0 0) 0x7fe74c0cee10 con 0x7fe75c074240 2024-03-16T10:09:15.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.202+0000 7fe759400640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 0x7fe7400a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.203+0000 7fe759400640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 0x7fe7400a4b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe748000bc0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:15.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.204+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe74c09b630 con 0x7fe75c074240 2024-03-16T10:09:15.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.549+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fe75c061d80 con 0x7fe7400a2660 2024-03-16T10:09:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.629+0000 7fe752a00640 1 -- 172.21.15.39:0/837169167 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fe75c061d80 con 0x7fe7400a2660 2024-03-16T10:09:15.630 INFO:teuthology.orchestra.run.smithi039.stdout:Retention added to path / 2024-03-16T10:09:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 msgr2=0x7fe7400a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 0x7fe7400a4b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe748000bc0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 msgr2=0x7fe75c19df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00e9f0 tx=0x7fe74c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 shutdown_connections 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10dc20 0x7fe75c1a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fe7400a2660 0x7fe7400a4b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe75c075820 0x7fe75c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 --2- 172.21.15.39:0/837169167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe75c074240 0x7fe75c19df70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 >> 172.21.15.39:0/837169167 conn(0x7fe75c0fcbc0 msgr2=0x7fe75c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 shutdown_connections 2024-03-16T10:09:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.631+0000 7fe75b200640 1 -- 172.21.15.39:0/837169167 wait complete. 2024-03-16T10:09:15.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 -- 172.21.15.39:0/2270073093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 msgr2=0x7fa724105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 --2- 172.21.15.39:0/2270073093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa724105640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009ca0 tx=0x7fa71c02c100 comp rx=0 tx=0).stop 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 -- 172.21.15.39:0/2270073093 shutdown_connections 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 --2- 172.21.15.39:0/2270073093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa724105640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 --2- 172.21.15.39:0/2270073093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa724104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 --2- 172.21.15.39:0/2270073093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa7241022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 -- 172.21.15.39:0/2270073093 >> 172.21.15.39:0/2270073093 conn(0x7fa7240fbac0 msgr2=0x7fa7240fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 -- 172.21.15.39:0/2270073093 shutdown_connections 2024-03-16T10:09:15.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.840+0000 7fa723e00640 1 -- 172.21.15.39:0/2270073093 wait complete. 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.841+0000 7fa723e00640 1 Processor -- start 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 -- start start 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa72419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa7241a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa724113880 con 0x7fa7240ffea0 2024-03-16T10:09:15.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa724113700 con 0x7fa7241051c0 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa724113a00 con 0x7fa724102810 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa722000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa722a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa72419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa722000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:36800/0 (socket says 172.21.15.39:36800) 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa722000640 1 -- 172.21.15.39:0/1140060598 learned_addr learned my addr 172.21.15.39:0/1140060598 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.842+0000 7fa723400640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa7241a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 -- 172.21.15.39:0/1140060598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 msgr2=0x7fa7241a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa7241a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 -- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 msgr2=0x7fa72419de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa72419de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:15.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 -- 172.21.15.39:0/1140060598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7241a5a60 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722a00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa72419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa722000640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa71000d950 tx=0x7fa71000dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710017030 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241a5d50 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa710004020 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710005650 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.843+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a6210 con 0x7fa724102810 2024-03-16T10:09:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.845+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7240fd5b0 con 0x7fa724102810 2024-03-16T10:09:15.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.847+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 23) ==== 137102+0+0 (secure 0 0 0) 0x7fa7100041d0 con 0x7fa724102810 2024-03-16T10:09:15.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.848+0000 7fa71b400640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 0x7fa7080a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:15.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.848+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9474+0+0 (secure 0 0 0) 0x7fa7100ce4b0 con 0x7fa724102810 2024-03-16T10:09:15.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.849+0000 7fa722a00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 0x7fa7080a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:15.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.849+0000 7fa722a00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 0x7fa7080a4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa71400afb0 tx=0x7fa71400a2d0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:15.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.853+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa71009b360 con 0x7fa724102810 2024-03-16T10:09:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:15.895+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 24) ==== 137410+0+0 (secure 0 0 0) 0x7fa7100972d0 con 0x7fa724102810 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: [16/Mar/2024:10:09:14] ENGINE Bus STARTING 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: mgrmap e23: x(active, since 1.02361s), standbys: y, z 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: [16/Mar/2024:10:09:14] ENGINE Serving on http://172.21.15.39:8765 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: from='client.25489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: [16/Mar/2024:10:09:15] ENGINE Serving on https://172.21.15.39:7150 2024-03-16T10:09:16.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:15 smithi039 ceph-mon[31510]: [16/Mar/2024:10:09:15] ENGINE Bus STARTED 2024-03-16T10:09:16.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.095+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa724061c20 con 0x7fa7080a25e0 2024-03-16T10:09:16.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.104+0000 7fa71b400640 1 -- 172.21.15.39:0/1140060598 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa724061c20 con 0x7fa7080a25e0 2024-03-16T10:09:16.106 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-16T00:00:00", "created": "2024-03-16T10:09:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-16T10:09:16.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 msgr2=0x7fa7080a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 0x7fa7080a4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa71400afb0 tx=0x7fa71400a2d0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 msgr2=0x7fa72419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa71000d950 tx=0x7fa71000dd50 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 shutdown_connections 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7241051c0 0x7fa7241a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fa7080a25e0 0x7fa7080a4aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa724102810 0x7fa72419e380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 --2- 172.21.15.39:0/1140060598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7240ffea0 0x7fa72419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 >> 172.21.15.39:0/1140060598 conn(0x7fa7240fbac0 msgr2=0x7fa724100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 shutdown_connections 2024-03-16T10:09:16.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.107+0000 7fa723e00640 1 -- 172.21.15.39:0/1140060598 wait complete. 2024-03-16T10:09:16.125 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: [16/Mar/2024:10:09:14] ENGINE Bus STARTING 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: mgrmap e23: x(active, since 1.02361s), standbys: y, z 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: [16/Mar/2024:10:09:14] ENGINE Serving on http://172.21.15.39:8765 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: from='client.25489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: [16/Mar/2024:10:09:15] ENGINE Serving on https://172.21.15.39:7150 2024-03-16T10:09:16.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:15 smithi194 ceph-mon[36445]: [16/Mar/2024:10:09:15] ENGINE Bus STARTED 2024-03-16T10:09:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 -- 172.21.15.39:0/2493569336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c1020b0 msgr2=0x7f0d5c1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 --2- 172.21.15.39:0/2493569336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c1044a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009ca0 tx=0x7f0d5002c0f0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 -- 172.21.15.39:0/2493569336 shutdown_connections 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 --2- 172.21.15.39:0/2493569336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c107310 0x7f0d5c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 --2- 172.21.15.39:0/2493569336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 --2- 172.21.15.39:0/2493569336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c1044a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 -- 172.21.15.39:0/2493569336 >> 172.21.15.39:0/2493569336 conn(0x7f0d5c0fbac0 msgr2=0x7f0d5c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 -- 172.21.15.39:0/2493569336 shutdown_connections 2024-03-16T10:09:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.341+0000 7f0d62600640 1 -- 172.21.15.39:0/2493569336 wait complete. 2024-03-16T10:09:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.342+0000 7f0d62600640 1 Processor -- start 2024-03-16T10:09:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.342+0000 7f0d62600640 1 -- start start 2024-03-16T10:09:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 0x7f0d5c1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d2c0 con 0x7f0d5c1020b0 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d140 con 0x7f0d5c107310 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d62600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d440 con 0x7f0d5c1049e0 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d5be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d5be00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:36832/0 (socket says 172.21.15.39:36832) 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d5be00640 1 -- 172.21.15.39:0/450660689 learned_addr learned my addr 172.21.15.39:0/450660689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d61c00640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 0x7f0d5c1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:16.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.343+0000 7f0d61200640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 -- 172.21.15.39:0/450660689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 msgr2=0x7f0d5c1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 0x7f0d5c1a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 -- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c1020b0 msgr2=0x7f0d5c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 -- 172.21.15.39:0/450660689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c1a1730 con 0x7f0d5c1049e0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d61c00640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 0x7f0d5c1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d5be00640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9f0 tx=0x7f0d4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800ccc0 con 0x7f0d5c1049e0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c1a1a20 con 0x7f0d5c1049e0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48005500 con 0x7f0d5c1049e0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48010690 con 0x7f0d5c1049e0 2024-03-16T10:09:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.344+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1a1f20 con 0x7f0d5c1049e0 2024-03-16T10:09:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.345+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c108e60 con 0x7f0d5c1049e0 2024-03-16T10:09:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.348+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 24) ==== 137410+0+0 (secure 0 0 0) 0x7f0d480046c0 con 0x7f0d5c1049e0 2024-03-16T10:09:16.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.349+0000 7f0d59600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 0x7f0d440a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:09:16.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.349+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9474+0+0 (secure 0 0 0) 0x7f0d480cf570 con 0x7f0d5c1049e0 2024-03-16T10:09:16.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.349+0000 7f0d61200640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 0x7f0d440a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:09:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.350+0000 7f0d61200640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 0x7f0d440a4e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d5002c700 tx=0x7f0d500323c0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:09:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.353+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0d4809bf10 con 0x7f0d5c1049e0 2024-03-16T10:09:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: [16/Mar/2024:10:09:14] ENGINE Bus STARTING 2024-03-16T10:09:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: mgrmap e23: x(active, since 1.02361s), standbys: y, z 2024-03-16T10:09:16.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: [16/Mar/2024:10:09:14] ENGINE Serving on http://172.21.15.39:8765 2024-03-16T10:09:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: from='client.25489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: [16/Mar/2024:10:09:15] ENGINE Serving on https://172.21.15.39:7150 2024-03-16T10:09:16.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:15 smithi105 ceph-mon[36559]: [16/Mar/2024:10:09:15] ENGINE Bus STARTED 2024-03-16T10:09:16.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.662+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0d5c061c20 con 0x7f0d440a2960 2024-03-16T10:09:16.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.670+0000 7f0d59600640 1 -- 172.21.15.39:0/450660689 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0d5c061c20 con 0x7f0d440a2960 2024-03-16T10:09:16.672 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.673+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 msgr2=0x7f0d440a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.673+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 0x7f0d440a4e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d5002c700 tx=0x7f0d500323c0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.673+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 msgr2=0x7f0d5c19a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.673+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9f0 tx=0x7f0d4800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 shutdown_connections 2024-03-16T10:09:16.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c107310 0x7f0d5c1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f0d440a2960 0x7f0d440a4e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d5c1049e0 0x7f0d5c19a050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 --2- 172.21.15.39:0/450660689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d5c1020b0 0x7f0d5c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 >> 172.21.15.39:0/450660689 conn(0x7f0d5c0fbac0 msgr2=0x7f0d5c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 shutdown_connections 2024-03-16T10:09:16.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:09:16.674+0000 7f0d62600640 1 -- 172.21.15.39:0/450660689 wait complete. 2024-03-16T10:09:16.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-16T10:09:16.783 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-16T10:09:16.795 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T10:09:16.904 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T10:09:16.904 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:09:16.904 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-03-16T10:09:16.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='client.25520 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:16.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:16.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-16T10:09:16.969 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:16.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:16.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:16.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:16.970 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:16 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.014 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:17.063 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem Size Used Avail Use% Mounted on 2024-03-16T10:09:17.063 INFO:teuthology.orchestra.run.smithi039.stdout:0@65a28618-e37b-11ee-95c9-87774f69a715.cephfs=/volumes/qa/sv_1/700e2c66-fa54-4588-bbc8-89eda93b2cfd 1.1T 352M 1.1T 1% /tmp/cephfs 2024-03-16T10:09:17.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='client.25520 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:16 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-03-16T10:09:17.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-03-16T10:09:17.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='client.25520 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:17.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='client.25528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:17.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:16 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='client.25540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:18.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:17 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='client.25540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.257 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:17 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='client.25540 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:18.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:17 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:19.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:18 smithi194 ceph-mon[36445]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:19.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:18 smithi039 ceph-mon[31510]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-16T10:09:19.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:09:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:09:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:09:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:09:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:09:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:18 smithi105 ceph-mon[36559]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-03-16T10:09:19.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-03-16T10:09:19.869 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-03-16T10:09:19.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-03-16T10:09:19.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 data 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:20.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:09:20.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-03-16T10:09:20.263 INFO:teuthology.orchestra.run.smithi039.stdout:[Service] 2024-03-16T10:09:20.263 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-03-16T10:09:20.263 INFO:teuthology.orchestra.run.smithi039.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-03-16T10:09:20.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-03-16T10:09:20.492 INFO:teuthology.orchestra.run.smithi039.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-03-16T10:09:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T10:09:20.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:20.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:20 smithi194 ceph-mon[36445]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:20.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:20 smithi039 ceph-mon[31510]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:20.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:20.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:20.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi039:/etc/ceph/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: pgmap v6: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:20.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:20 smithi105 ceph-mon[36559]: Updating smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:21 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:21 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:21 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:21 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:21 smithi105 ceph-mon[36559]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:21 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:21 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:21 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:21 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:21.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:21 smithi194 ceph-mon[36445]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:21 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:21 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:21 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:21 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:22.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:21 smithi039 ceph-mon[31510]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:09:23.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:23.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 81 op/s 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:23.366 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:23 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 81 op/s 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:23 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: Updating smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 133 KiB/s wr, 81 op/s 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:09:23.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:23 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:09:25.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:25 smithi105 ceph-mon[36559]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 104 KiB/s wr, 63 op/s 2024-03-16T10:09:25.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:25 smithi039 ceph-mon[31510]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 104 KiB/s wr, 63 op/s 2024-03-16T10:09:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:25 smithi194 ceph-mon[36445]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 104 KiB/s wr, 63 op/s 2024-03-16T10:09:27.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:27 smithi105 ceph-mon[36559]: pgmap v9: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.4 MiB/s wr, 85 op/s 2024-03-16T10:09:27.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:27 smithi194 ceph-mon[36445]: pgmap v9: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.4 MiB/s wr, 85 op/s 2024-03-16T10:09:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:27 smithi039 ceph-mon[31510]: pgmap v9: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 2.4 MiB/s wr, 85 op/s 2024-03-16T10:09:29.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:29 smithi105 ceph-mon[36559]: pgmap v10: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-16T10:09:29.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:29 smithi194 ceph-mon[36445]: pgmap v10: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-16T10:09:29.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:29 smithi039 ceph-mon[31510]: pgmap v10: 129 pgs: 129 active+clean; 27 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 2.2 MiB/s wr, 77 op/s 2024-03-16T10:09:30.509 INFO:teuthology.orchestra.run.smithi039.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-03-16T10:09:30.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 data 2024-03-16T10:09:30.671 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:total 35 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 . 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 base 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 16 10:09 global 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 log 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_commit_ts 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_dynshmem 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 16 10:09 pg_hba.conf 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 16 10:09 pg_ident.conf 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 pg_logical 2024-03-16T10:09:30.672 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 pg_multixact 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_notify 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_replslot 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_serial 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_snapshots 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_stat 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_stat_tmp 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 pg_subtrans 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_tblspc 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_twophase 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 pg_wal 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 pg_xact 2024-03-16T10:09:30.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 16 10:09 postgresql.auto.conf 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 16 10:09 postgresql.conf 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 . 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 1 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 13493 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 13494 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:total 7777 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 . 2024-03-16T10:09:30.674 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 .. 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:09 1259 2024-03-16T10:09:30.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:09:30.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:09:30.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:09:30.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:09:30.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:09:30.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:09 2619 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2619_fsm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:09:30.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:09:30.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:09:30.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2696 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:09:30.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:09:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:09:30.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:09:30.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:09:30.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:09:30.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:09:30.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:09:30.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:09:30.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:09:30.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:09:30.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:09:30.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:total 7777 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 . 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 .. 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:09:30.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:09 1259 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:09:30.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:09:30.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:09:30.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:09:30.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:09:30.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:09 2619 2024-03-16T10:09:30.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2619_fsm 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:09:30.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:09:30.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2696 2024-03-16T10:09:30.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:09:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:09:30.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:09:30.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:09:30.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:09:30.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:09:30.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:09:30.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:09:30.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:09:30.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:total 7777 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 . 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 .. 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:09:30.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:09 1259 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:09:30.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:09:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:09:30.719 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:09:30.720 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:09:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:09:30.722 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:09 2619 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2619_fsm 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:09:30.723 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:09:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:09:30.725 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2696 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:09:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:09:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:09:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:09:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:09:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:09:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:09:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:09:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:09:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:09:30.735 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:09:30.738 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:09:30.739 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:09:30.739 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:total 537 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 16 10:09 . 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1213 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1213_fsm 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1213_vm 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1214 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1214_fsm 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1214_vm 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 1232 2024-03-16T10:09:30.740 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 1233 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1260 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1260_fsm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1260_vm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1261 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1261_fsm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1261_vm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1262 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1262_fsm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1262_vm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2396 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2396_fsm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2396_vm 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2397 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2671 2024-03-16T10:09:30.741 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2672 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2676 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2677 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2694 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2695 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2697 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2698 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2846 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2847 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2964 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2965 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2966 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2967 2024-03-16T10:09:30.742 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3592 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3593 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4060 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4061 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4175 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4176 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4177 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4178 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4181 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4182 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4183 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4184 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4185 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4186 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6000 2024-03-16T10:09:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6001 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6002 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6100 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6114 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6115 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 pg_control 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 . 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 mappings 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 16 10:09 replorigin_checkpoint 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 snapshots 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-16T10:09:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 .. 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 .. 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 . 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 members 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 offsets 2024-03-16T10:09:30.746 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 .. 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 0000 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 .. 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 0000 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-16T10:09:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-16T10:09:30.748 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-16T10:09:30.749 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 0000 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.750 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:total 16384 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 . 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:09 000000010000000000000001 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 archive_status 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 .. 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:09:30.751 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 16 10:09 .. 2024-03-16T10:09:30.752 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 0000 2024-03-16T10:09:30.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-03-16T10:09:31.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:31 smithi039 ceph-mon[31510]: pgmap v11: 129 pgs: 129 active+clean; 38 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.5 MiB/s wr, 128 op/s 2024-03-16T10:09:31.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-03-16T10:09:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:dropping old tables... 2024-03-16T10:09:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-03-16T10:09:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-03-16T10:09:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-03-16T10:09:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-03-16T10:09:31.237 INFO:teuthology.orchestra.run.smithi039.stderr:creating tables... 2024-03-16T10:09:31.262 INFO:teuthology.orchestra.run.smithi039.stderr:generating data (client-side)... 2024-03-16T10:09:31.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:31 smithi105 ceph-mon[36559]: pgmap v11: 129 pgs: 129 active+clean; 38 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.5 MiB/s wr, 128 op/s 2024-03-16T10:09:31.398 INFO:teuthology.orchestra.run.smithi039.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.10 s, remaining 3.12 s) 2024-03-16T10:09:31.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:31 smithi194 ceph-mon[36445]: pgmap v11: 129 pgs: 129 active+clean; 38 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.5 MiB/s wr, 128 op/s 2024-03-16T10:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.45 s, remaining 6.71 s) 2024-03-16T10:09:31.891 INFO:teuthology.orchestra.run.smithi039.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.59 s, remaining 5.74 s) 2024-03-16T10:09:32.224 INFO:teuthology.orchestra.run.smithi039.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.93 s, remaining 6.49 s) 2024-03-16T10:09:32.597 INFO:teuthology.orchestra.run.smithi039.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.30 s, remaining 7.02 s) 2024-03-16T10:09:32.953 INFO:teuthology.orchestra.run.smithi039.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.66 s, remaining 7.18 s) 2024-03-16T10:09:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.77 s, remaining 6.33 s) 2024-03-16T10:09:33.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:33 smithi105 ceph-mon[36559]: pgmap v12: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 4.1 MiB/s wr, 138 op/s 2024-03-16T10:09:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.12 s, remaining 6.35 s) 2024-03-16T10:09:33.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:33 smithi194 ceph-mon[36445]: pgmap v12: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 4.1 MiB/s wr, 138 op/s 2024-03-16T10:09:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:33 smithi039 ceph-mon[31510]: pgmap v12: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 4.1 MiB/s wr, 138 op/s 2024-03-16T10:09:33.705 INFO:teuthology.orchestra.run.smithi039.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.41 s, remaining 6.15 s) 2024-03-16T10:09:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.50 s, remaining 5.50 s) 2024-03-16T10:09:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.96 s, remaining 5.64 s) 2024-03-16T10:09:34.370 INFO:teuthology.orchestra.run.smithi039.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.07 s, remaining 5.12 s) 2024-03-16T10:09:34.761 INFO:teuthology.orchestra.run.smithi039.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.46 s, remaining 5.06 s) 2024-03-16T10:09:35.178 INFO:teuthology.orchestra.run.smithi039.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.88 s, remaining 4.99 s) 2024-03-16T10:09:35.282 INFO:teuthology.orchestra.run.smithi039.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.98 s, remaining 4.52 s) 2024-03-16T10:09:35.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:35 smithi105 ceph-mon[36559]: pgmap v13: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.1 MiB/s wr, 98 op/s 2024-03-16T10:09:35.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:35 smithi194 ceph-mon[36445]: pgmap v13: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.1 MiB/s wr, 98 op/s 2024-03-16T10:09:35.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:35 smithi039 ceph-mon[31510]: pgmap v13: 129 pgs: 129 active+clean; 43 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 4.1 MiB/s wr, 98 op/s 2024-03-16T10:09:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.37 s, remaining 4.37 s) 2024-03-16T10:09:35.841 INFO:teuthology.orchestra.run.smithi039.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.54 s, remaining 4.01 s) 2024-03-16T10:09:36.152 INFO:teuthology.orchestra.run.smithi039.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.86 s, remaining 3.78 s) 2024-03-16T10:09:36.456 INFO:teuthology.orchestra.run.smithi039.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.16 s, remaining 3.53 s) 2024-03-16T10:09:36.617 INFO:teuthology.orchestra.run.smithi039.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.32 s, remaining 3.19 s) 2024-03-16T10:09:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.60 s, remaining 2.93 s) 2024-03-16T10:09:37.213 INFO:teuthology.orchestra.run.smithi039.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.92 s, remaining 2.69 s) 2024-03-16T10:09:37.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:37 smithi105 ceph-mon[36559]: pgmap v14: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 26 MiB/s wr, 167 op/s 2024-03-16T10:09:37.365 INFO:teuthology.orchestra.run.smithi039.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.07 s, remaining 2.37 s) 2024-03-16T10:09:37.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:37 smithi194 ceph-mon[36445]: pgmap v14: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 26 MiB/s wr, 167 op/s 2024-03-16T10:09:37.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:37 smithi039 ceph-mon[31510]: pgmap v14: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 26 MiB/s wr, 167 op/s 2024-03-16T10:09:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.37 s, remaining 2.12 s) 2024-03-16T10:09:37.969 INFO:teuthology.orchestra.run.smithi039.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.67 s, remaining 1.87 s) 2024-03-16T10:09:38.122 INFO:teuthology.orchestra.run.smithi039.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.82 s, remaining 1.57 s) 2024-03-16T10:09:38.404 INFO:teuthology.orchestra.run.smithi039.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.11 s, remaining 1.32 s) 2024-03-16T10:09:38.611 INFO:teuthology.orchestra.run.smithi039.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.31 s, remaining 1.04 s) 2024-03-16T10:09:38.906 INFO:teuthology.orchestra.run.smithi039.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.61 s, remaining 0.79 s) 2024-03-16T10:09:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.95 s, remaining 0.53 s) 2024-03-16T10:09:39.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:39 smithi105 ceph-mon[36559]: pgmap v15: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 24 MiB/s wr, 136 op/s 2024-03-16T10:09:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:3100000 of 3200000 tuples (96%) done (elapsed 8.13 s, remaining 0.26 s) 2024-03-16T10:09:39.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:39 smithi194 ceph-mon[36445]: pgmap v15: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 24 MiB/s wr, 136 op/s 2024-03-16T10:09:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:39 smithi039 ceph-mon[31510]: pgmap v15: 129 pgs: 129 active+clean; 173 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 24 MiB/s wr, 136 op/s 2024-03-16T10:09:39.780 INFO:teuthology.orchestra.run.smithi039.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.48 s, remaining 0.00 s) 2024-03-16T10:09:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:vacuuming... 2024-03-16T10:09:40.852 INFO:teuthology.orchestra.run.smithi039.stderr:creating primary keys... 2024-03-16T10:09:41.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:41 smithi105 ceph-mon[36559]: pgmap v16: 129 pgs: 129 active+clean; 333 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 51 MiB/s wr, 156 op/s 2024-03-16T10:09:41.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:41 smithi194 ceph-mon[36445]: pgmap v16: 129 pgs: 129 active+clean; 333 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 51 MiB/s wr, 156 op/s 2024-03-16T10:09:41.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:41 smithi039 ceph-mon[31510]: pgmap v16: 129 pgs: 129 active+clean; 333 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 51 MiB/s wr, 156 op/s 2024-03-16T10:09:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:done in 12.05 s (drop tables 0.00 s, create tables 0.03 s, client-side generate 8.58 s, vacuum 1.01 s, primary keys 2.44 s). 2024-03-16T10:09:43.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-03-16T10:09:43.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:43 smithi105 ceph-mon[36559]: pgmap v17: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 55 MiB/s wr, 109 op/s 2024-03-16T10:09:43.365 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:43 smithi039 ceph-mon[31510]: pgmap v17: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 55 MiB/s wr, 109 op/s 2024-03-16T10:09:43.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:43 smithi194 ceph-mon[36445]: pgmap v17: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 55 MiB/s wr, 109 op/s 2024-03-16T10:09:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:starting vacuum...end. 2024-03-16T10:09:45.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:45 smithi105 ceph-mon[36559]: pgmap v18: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 54 MiB/s wr, 92 op/s 2024-03-16T10:09:45.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:45 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:45.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:45 smithi194 ceph-mon[36445]: pgmap v18: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 54 MiB/s wr, 92 op/s 2024-03-16T10:09:45.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:45 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:45 smithi039 ceph-mon[31510]: pgmap v18: 129 pgs: 129 active+clean; 365 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 54 MiB/s wr, 92 op/s 2024-03-16T10:09:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:45 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:47.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:47 smithi105 ceph-mon[36559]: pgmap v19: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 72 MiB/s wr, 129 op/s 2024-03-16T10:09:47.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:47 smithi194 ceph-mon[36445]: pgmap v19: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 72 MiB/s wr, 129 op/s 2024-03-16T10:09:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:47 smithi039 ceph-mon[31510]: pgmap v19: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 72 MiB/s wr, 129 op/s 2024-03-16T10:09:48.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 5.0 s, 2190.2 tps, lat 43.310 ms stddev 47.998 2024-03-16T10:09:49.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:49 smithi194 ceph-mon[36445]: pgmap v20: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 50 MiB/s wr, 59 op/s 2024-03-16T10:09:49.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:49 smithi039 ceph-mon[31510]: pgmap v20: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 50 MiB/s wr, 59 op/s 2024-03-16T10:09:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:49 smithi105 ceph-mon[36559]: pgmap v20: 129 pgs: 129 active+clean; 514 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 50 MiB/s wr, 59 op/s 2024-03-16T10:09:50.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:50 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:50.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:50 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:50.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:50 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:51.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:51 smithi194 ceph-mon[36445]: pgmap v21: 129 pgs: 129 active+clean; 832 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 52 MiB/s wr, 206 op/s 2024-03-16T10:09:51.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:51 smithi039 ceph-mon[31510]: pgmap v21: 129 pgs: 129 active+clean; 832 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 52 MiB/s wr, 206 op/s 2024-03-16T10:09:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:51 smithi105 ceph-mon[36559]: pgmap v21: 129 pgs: 129 active+clean; 832 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 52 MiB/s wr, 206 op/s 2024-03-16T10:09:53.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:53 smithi194 ceph-mon[36445]: pgmap v22: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 195 op/s 2024-03-16T10:09:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 10.0 s, 1910.8 tps, lat 52.201 ms stddev 57.570 2024-03-16T10:09:53.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:53 smithi039 ceph-mon[31510]: pgmap v22: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 195 op/s 2024-03-16T10:09:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:53 smithi105 ceph-mon[36559]: pgmap v22: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 25 MiB/s wr, 195 op/s 2024-03-16T10:09:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:55 smithi194 ceph-mon[36445]: pgmap v23: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 192 op/s 2024-03-16T10:09:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:55 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:55.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:55 smithi039 ceph-mon[31510]: pgmap v23: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 192 op/s 2024-03-16T10:09:55.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:55 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:55 smithi105 ceph-mon[36559]: pgmap v23: 129 pgs: 129 active+clean; 870 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 192 op/s 2024-03-16T10:09:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:55 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:09:57.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:57 smithi194 ceph-mon[36445]: pgmap v24: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 464 op/s 2024-03-16T10:09:57.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:57 smithi039 ceph-mon[31510]: pgmap v24: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 464 op/s 2024-03-16T10:09:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:57 smithi105 ceph-mon[36559]: pgmap v24: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 464 op/s 2024-03-16T10:09:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 15.0 s, 1894.2 tps, lat 51.690 ms stddev 58.371 2024-03-16T10:09:59.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:09:59 smithi194 ceph-mon[36445]: pgmap v25: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 426 op/s 2024-03-16T10:09:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:09:59 smithi039 ceph-mon[31510]: pgmap v25: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 426 op/s 2024-03-16T10:09:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:09:59 smithi105 ceph-mon[36559]: pgmap v25: 129 pgs: 129 active+clean; 912 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 426 op/s 2024-03-16T10:10:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:00 smithi194 ceph-mon[36445]: overall HEALTH_OK 2024-03-16T10:10:00.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-mon[31510]: overall HEALTH_OK 2024-03-16T10:10:00.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:00 smithi105 ceph-mon[36559]: overall HEALTH_OK 2024-03-16T10:10:01.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:10:00.693+0000 7f7137200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:10:01.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:10:00.693+0000 7f7137200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:10:01.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:10:00.693+0000 7f7137200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:10:01.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:10:00.693+0000 7f7137200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:10:01.005 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:10:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:10:00.693+0000 7f7137200700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T10:10:01.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:01 smithi194 ceph-mon[36445]: pgmap v26: 129 pgs: 129 active+clean; 930 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.8 MiB/s wr, 679 op/s 2024-03-16T10:10:01.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:01 smithi039 ceph-mon[31510]: pgmap v26: 129 pgs: 129 active+clean; 930 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.8 MiB/s wr, 679 op/s 2024-03-16T10:10:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:01 smithi105 ceph-mon[36559]: pgmap v26: 129 pgs: 129 active+clean; 930 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.8 MiB/s wr, 679 op/s 2024-03-16T10:10:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:03 smithi194 ceph-mon[36445]: pgmap v27: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 548 op/s 2024-03-16T10:10:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:03 smithi194 ceph-mon[36445]: mgrmap e26: x(active, since 48s), standbys: y, z 2024-03-16T10:10:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 20.0 s, 1219.2 tps, lat 77.954 ms stddev 162.380 2024-03-16T10:10:03.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:03 smithi039 ceph-mon[31510]: pgmap v27: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 548 op/s 2024-03-16T10:10:03.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:03 smithi039 ceph-mon[31510]: mgrmap e26: x(active, since 48s), standbys: y, z 2024-03-16T10:10:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:03 smithi105 ceph-mon[36559]: pgmap v27: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 548 op/s 2024-03-16T10:10:03.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:03 smithi105 ceph-mon[36559]: mgrmap e26: x(active, since 48s), standbys: y, z 2024-03-16T10:10:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:05 smithi194 ceph-mon[36445]: pgmap v28: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-03-16T10:10:05.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:05 smithi039 ceph-mon[31510]: pgmap v28: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-03-16T10:10:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:05 smithi105 ceph-mon[36559]: pgmap v28: 129 pgs: 129 active+clean; 935 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 539 op/s 2024-03-16T10:10:07.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:07 smithi194 ceph-mon[36445]: pgmap v29: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-16T10:10:07.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:07 smithi039 ceph-mon[31510]: pgmap v29: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-16T10:10:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:07 smithi105 ceph-mon[36559]: pgmap v29: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-16T10:10:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 25.0 s, 138.4 tps, lat 691.396 ms stddev 430.678 2024-03-16T10:10:09.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:09 smithi194 ceph-mon[36445]: pgmap v30: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-03-16T10:10:09.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:09 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:10:09.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:09 smithi039 ceph-mon[31510]: pgmap v30: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-03-16T10:10:09.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:09 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:10:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:09 smithi105 ceph-mon[36559]: pgmap v30: 129 pgs: 129 active+clean; 938 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.3 MiB/s wr, 441 op/s 2024-03-16T10:10:09.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:09 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:10:11.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:11 smithi039 ceph-mon[31510]: pgmap v31: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-03-16T10:10:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:11 smithi105 ceph-mon[36559]: pgmap v31: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-03-16T10:10:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:11 smithi194 ceph-mon[36445]: pgmap v31: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 624 op/s 2024-03-16T10:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 30.0 s, 212.4 tps, lat 517.556 ms stddev 856.491 2024-03-16T10:10:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:13 smithi039 ceph-mon[31510]: pgmap v32: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-03-16T10:10:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:13 smithi105 ceph-mon[36559]: pgmap v32: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-03-16T10:10:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:13 smithi194 ceph-mon[36445]: pgmap v32: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-03-16T10:10:15.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:15 smithi039 ceph-mon[31510]: pgmap v33: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-03-16T10:10:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:15 smithi105 ceph-mon[36559]: pgmap v33: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-03-16T10:10:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:15 smithi194 ceph-mon[36445]: pgmap v33: 129 pgs: 129 active+clean; 959 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.1 MiB/s wr, 356 op/s 2024-03-16T10:10:17.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:17 smithi039 ceph-mon[31510]: pgmap v34: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 1.05k op/s 2024-03-16T10:10:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:17 smithi105 ceph-mon[36559]: pgmap v34: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 1.05k op/s 2024-03-16T10:10:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:17 smithi194 ceph-mon[36445]: pgmap v34: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 41 MiB/s wr, 1.05k op/s 2024-03-16T10:10:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 35.0 s, 1452.4 tps, lat 69.047 ms stddev 82.442 2024-03-16T10:10:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:19 smithi039 ceph-mon[31510]: pgmap v35: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 878 op/s 2024-03-16T10:10:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:19 smithi105 ceph-mon[36559]: pgmap v35: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 878 op/s 2024-03-16T10:10:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:19 smithi194 ceph-mon[36445]: pgmap v35: 129 pgs: 129 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 878 op/s 2024-03-16T10:10:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:21 smithi105 ceph-mon[36559]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.07k op/s 2024-03-16T10:10:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:21 smithi194 ceph-mon[36445]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.07k op/s 2024-03-16T10:10:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:21 smithi039 ceph-mon[31510]: pgmap v36: 129 pgs: 129 active+clean; 989 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 41 MiB/s wr, 1.07k op/s 2024-03-16T10:10:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 40.0 s, 1701.2 tps, lat 59.040 ms stddev 71.372 2024-03-16T10:10:23.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:23 smithi105 ceph-mon[36559]: pgmap v37: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 946 op/s 2024-03-16T10:10:23.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:23 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:10:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:23 smithi194 ceph-mon[36445]: pgmap v37: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 946 op/s 2024-03-16T10:10:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:23 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:10:23.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:23 smithi039 ceph-mon[31510]: pgmap v37: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 946 op/s 2024-03-16T10:10:23.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:23 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:10:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:25 smithi105 ceph-mon[36559]: pgmap v38: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 947 op/s 2024-03-16T10:10:25.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:25 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:10:25.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:25 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:10:25.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:25 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:10:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:25 smithi194 ceph-mon[36445]: pgmap v38: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 947 op/s 2024-03-16T10:10:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:25 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:10:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:25 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:10:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:25 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:10:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:25 smithi039 ceph-mon[31510]: pgmap v38: 129 pgs: 129 active+clean; 986 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 MiB/s wr, 947 op/s 2024-03-16T10:10:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:25 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:10:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:25 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:10:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:25 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:10:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:27 smithi105 ceph-mon[36559]: pgmap v39: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 1.20k op/s 2024-03-16T10:10:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:27 smithi194 ceph-mon[36445]: pgmap v39: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 1.20k op/s 2024-03-16T10:10:27.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:27 smithi039 ceph-mon[31510]: pgmap v39: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 40 MiB/s wr, 1.20k op/s 2024-03-16T10:10:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 45.0 s, 1686.8 tps, lat 59.379 ms stddev 68.321 2024-03-16T10:10:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:29 smithi105 ceph-mon[36559]: pgmap v40: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-03-16T10:10:29.705 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:29 smithi039 ceph-mon[31510]: pgmap v40: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-03-16T10:10:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:29 smithi194 ceph-mon[36445]: pgmap v40: 129 pgs: 129 active+clean; 1008 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-03-16T10:10:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:31 smithi105 ceph-mon[36559]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-03-16T10:10:31.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:31 smithi194 ceph-mon[36445]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-03-16T10:10:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:31 smithi039 ceph-mon[31510]: pgmap v41: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 7.5 MiB/s wr, 724 op/s 2024-03-16T10:10:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 50.0 s, 1431.6 tps, lat 69.464 ms stddev 101.965 2024-03-16T10:10:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:33 smithi105 ceph-mon[36559]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-03-16T10:10:33.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:33 smithi194 ceph-mon[36445]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-03-16T10:10:33.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:33 smithi039 ceph-mon[31510]: pgmap v42: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-03-16T10:10:34.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:34 smithi105 ceph-mon[36559]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-03-16T10:10:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:34 smithi194 ceph-mon[36445]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-03-16T10:10:34.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:34 smithi039 ceph-mon[31510]: pgmap v43: 129 pgs: 129 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-03-16T10:10:37.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:37 smithi105 ceph-mon[36559]: pgmap v44: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 8.5 MiB/s wr, 778 op/s 2024-03-16T10:10:37.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:37 smithi194 ceph-mon[36445]: pgmap v44: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 8.5 MiB/s wr, 778 op/s 2024-03-16T10:10:37.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:37 smithi039 ceph-mon[31510]: pgmap v44: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 8.5 MiB/s wr, 778 op/s 2024-03-16T10:10:38.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 55.0 s, 2808.0 tps, lat 36.126 ms stddev 42.631 2024-03-16T10:10:39.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:39 smithi105 ceph-mon[36559]: pgmap v45: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 6.2 MiB/s wr, 525 op/s 2024-03-16T10:10:39.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:39 smithi194 ceph-mon[36445]: pgmap v45: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 6.2 MiB/s wr, 525 op/s 2024-03-16T10:10:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:39 smithi039 ceph-mon[31510]: pgmap v45: 129 pgs: 129 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 6.2 MiB/s wr, 525 op/s 2024-03-16T10:10:41.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:41 smithi105 ceph-mon[36559]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 9.2 MiB/s wr, 824 op/s 2024-03-16T10:10:41.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:41 smithi194 ceph-mon[36445]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 9.2 MiB/s wr, 824 op/s 2024-03-16T10:10:41.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:41 smithi039 ceph-mon[31510]: pgmap v46: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 9.2 MiB/s wr, 824 op/s 2024-03-16T10:10:43.309 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:43 smithi039 ceph-mon[31510]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-16T10:10:43.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:43 smithi105 ceph-mon[36559]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-16T10:10:43.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:43 smithi194 ceph-mon[36445]: pgmap v47: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.6 MiB/s wr, 641 op/s 2024-03-16T10:10:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 60.0 s, 2694.4 tps, lat 37.018 ms stddev 41.305 2024-03-16T10:10:45.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:45 smithi039 ceph-mon[31510]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-16T10:10:45.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:45 smithi105 ceph-mon[36559]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-16T10:10:45.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:45 smithi194 ceph-mon[36445]: pgmap v48: 129 pgs: 129 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-16T10:10:47.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:47 smithi105 ceph-mon[36559]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 9.8 MiB/s wr, 885 op/s 2024-03-16T10:10:47.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:47 smithi105 ceph-mon[36559]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-16T10:10:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:47 smithi194 ceph-mon[36445]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 9.8 MiB/s wr, 885 op/s 2024-03-16T10:10:47.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:47 smithi194 ceph-mon[36445]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-16T10:10:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:47 smithi039 ceph-mon[31510]: pgmap v49: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 9.8 MiB/s wr, 885 op/s 2024-03-16T10:10:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:47 smithi039 ceph-mon[31510]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-03-16T10:10:48.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 65.0 s, 1781.8 tps, lat 56.201 ms stddev 64.975 2024-03-16T10:10:49.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:49 smithi105 ceph-mon[36559]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.3 MiB/s wr, 617 op/s 2024-03-16T10:10:49.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:49 smithi194 ceph-mon[36445]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.3 MiB/s wr, 617 op/s 2024-03-16T10:10:49.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:49 smithi039 ceph-mon[31510]: pgmap v50: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.3 MiB/s wr, 617 op/s 2024-03-16T10:10:51.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:51 smithi105 ceph-mon[36559]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 825 op/s 2024-03-16T10:10:51.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:51 smithi194 ceph-mon[36445]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 825 op/s 2024-03-16T10:10:51.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:51 smithi039 ceph-mon[31510]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.8 MiB/s wr, 825 op/s 2024-03-16T10:10:53.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:53 smithi105 ceph-mon[36559]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-03-16T10:10:53.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:53 smithi194 ceph-mon[36445]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-03-16T10:10:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 70.0 s, 1825.6 tps, lat 54.605 ms stddev 67.810 2024-03-16T10:10:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:53 smithi039 ceph-mon[31510]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.5 MiB/s wr, 592 op/s 2024-03-16T10:10:55.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:55 smithi105 ceph-mon[36559]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-03-16T10:10:55.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:55 smithi194 ceph-mon[36445]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-03-16T10:10:55.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:55 smithi039 ceph-mon[31510]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 555 op/s 2024-03-16T10:10:57.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:57 smithi039 ceph-mon[31510]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 810 op/s 2024-03-16T10:10:57.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:57 smithi105 ceph-mon[36559]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 810 op/s 2024-03-16T10:10:57.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:57 smithi194 ceph-mon[36445]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.5 MiB/s wr, 810 op/s 2024-03-16T10:10:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 75.0 s, 1414.2 tps, lat 70.436 ms stddev 97.461 2024-03-16T10:10:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:10:59 smithi105 ceph-mon[36559]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-03-16T10:10:59.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:10:59 smithi194 ceph-mon[36445]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-03-16T10:10:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:10:59 smithi039 ceph-mon[31510]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-03-16T10:11:01.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:01 smithi105 ceph-mon[36559]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 703 op/s 2024-03-16T10:11:01.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:01 smithi194 ceph-mon[36445]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 703 op/s 2024-03-16T10:11:01.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:01 smithi039 ceph-mon[31510]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.3 MiB/s wr, 703 op/s 2024-03-16T10:11:03.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:03 smithi105 ceph-mon[36559]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-03-16T10:11:03.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:03 smithi194 ceph-mon[36445]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-03-16T10:11:03.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 80.0 s, 761.6 tps, lat 125.781 ms stddev 180.947 2024-03-16T10:11:03.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:03 smithi039 ceph-mon[31510]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 609 op/s 2024-03-16T10:11:05.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:05 smithi105 ceph-mon[36559]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-03-16T10:11:05.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:05 smithi194 ceph-mon[36445]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-03-16T10:11:05.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:05 smithi039 ceph-mon[31510]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-03-16T10:11:07.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:07 smithi105 ceph-mon[36559]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.4 MiB/s wr, 703 op/s 2024-03-16T10:11:07.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:07 smithi194 ceph-mon[36445]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.4 MiB/s wr, 703 op/s 2024-03-16T10:11:07.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:07 smithi039 ceph-mon[31510]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.4 MiB/s wr, 703 op/s 2024-03-16T10:11:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 85.0 s, 196.0 tps, lat 511.666 ms stddev 294.788 2024-03-16T10:11:09.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:09 smithi105 ceph-mon[36559]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-16T10:11:09.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:09 smithi194 ceph-mon[36445]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-16T10:11:09.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:09 smithi039 ceph-mon[31510]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.0 MiB/s wr, 448 op/s 2024-03-16T10:11:11.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:11 smithi194 ceph-mon[36445]: pgmap v61: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-03-16T10:11:11.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:11 smithi039 ceph-mon[31510]: pgmap v61: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-03-16T10:11:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:11 smithi105 ceph-mon[36559]: pgmap v61: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-03-16T10:11:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:13 smithi194 ceph-mon[36445]: pgmap v62: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 404 op/s 2024-03-16T10:11:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 90.0 s, 115.8 tps, lat 738.007 ms stddev 516.565 2024-03-16T10:11:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:13 smithi039 ceph-mon[31510]: pgmap v62: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 404 op/s 2024-03-16T10:11:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:13 smithi105 ceph-mon[36559]: pgmap v62: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.2 MiB/s wr, 404 op/s 2024-03-16T10:11:15.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:15 smithi194 ceph-mon[36445]: pgmap v63: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-03-16T10:11:15.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:15 smithi039 ceph-mon[31510]: pgmap v63: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-03-16T10:11:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:15 smithi105 ceph-mon[36559]: pgmap v63: 129 pgs: 129 active+clean; 1.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-03-16T10:11:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:17 smithi194 ceph-mon[36445]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-03-16T10:11:17.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:17 smithi039 ceph-mon[31510]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-03-16T10:11:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:17 smithi105 ceph-mon[36559]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 1.55k op/s 2024-03-16T10:11:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 95.0 s, 729.4 tps, lat 162.483 ms stddev 402.055 2024-03-16T10:11:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:19 smithi194 ceph-mon[36445]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 1.39k op/s 2024-03-16T10:11:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:19 smithi039 ceph-mon[31510]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 1.39k op/s 2024-03-16T10:11:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:19 smithi105 ceph-mon[36559]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 1.39k op/s 2024-03-16T10:11:21.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:21 smithi194 ceph-mon[36445]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 1.51k op/s 2024-03-16T10:11:21.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:21 smithi039 ceph-mon[31510]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 1.51k op/s 2024-03-16T10:11:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:21 smithi105 ceph-mon[36559]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 MiB/s wr, 1.51k op/s 2024-03-16T10:11:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:23 smithi194 ceph-mon[36445]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-03-16T10:11:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 100.0 s, 1592.9 tps, lat 62.926 ms stddev 70.071 2024-03-16T10:11:23.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:23 smithi039 ceph-mon[31510]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-03-16T10:11:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:23 smithi105 ceph-mon[36559]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 28 MiB/s wr, 1.58k op/s 2024-03-16T10:11:25.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:25 smithi194 ceph-mon[36445]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-03-16T10:11:25.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:25 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:11:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:25 smithi039 ceph-mon[31510]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-03-16T10:11:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:25 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:11:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:25 smithi105 ceph-mon[36559]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 MiB/s wr, 1.51k op/s 2024-03-16T10:11:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:25 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:11:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:27 smithi194 ceph-mon[36445]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.76k op/s 2024-03-16T10:11:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:27 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:11:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:27 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:11:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:27 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:11:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:27 smithi039 ceph-mon[31510]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.76k op/s 2024-03-16T10:11:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:27 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:11:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:27 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:11:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:27 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:11:27.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:27 smithi105 ceph-mon[36559]: pgmap v69: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.76k op/s 2024-03-16T10:11:27.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:27 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:11:27.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:27 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:11:27.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:27 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:11:28.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 105.0 s, 1492.6 tps, lat 66.398 ms stddev 75.106 2024-03-16T10:11:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:29 smithi194 ceph-mon[36445]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 509 op/s 2024-03-16T10:11:29.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:29 smithi039 ceph-mon[31510]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 509 op/s 2024-03-16T10:11:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:29 smithi105 ceph-mon[36559]: pgmap v70: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.8 MiB/s wr, 509 op/s 2024-03-16T10:11:31.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:31 smithi194 ceph-mon[36445]: pgmap v71: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 599 op/s 2024-03-16T10:11:31.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:31 smithi039 ceph-mon[31510]: pgmap v71: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 599 op/s 2024-03-16T10:11:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:31 smithi105 ceph-mon[36559]: pgmap v71: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 599 op/s 2024-03-16T10:11:33.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:33 smithi194 ceph-mon[36445]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-03-16T10:11:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 110.0 s, 1427.8 tps, lat 70.491 ms stddev 91.339 2024-03-16T10:11:33.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:33 smithi039 ceph-mon[31510]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-03-16T10:11:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:33 smithi105 ceph-mon[36559]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 625 op/s 2024-03-16T10:11:35.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:35 smithi194 ceph-mon[36445]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-03-16T10:11:35.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:35 smithi039 ceph-mon[31510]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-03-16T10:11:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:35 smithi105 ceph-mon[36559]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 MiB/s wr, 499 op/s 2024-03-16T10:11:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:37 smithi194 ceph-mon[36445]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-03-16T10:11:37.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:37 smithi039 ceph-mon[31510]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-03-16T10:11:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:37 smithi105 ceph-mon[36559]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.6 MiB/s wr, 751 op/s 2024-03-16T10:11:38.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 115.0 s, 1415.4 tps, lat 70.390 ms stddev 81.799 2024-03-16T10:11:39.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:39 smithi194 ceph-mon[36445]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 495 op/s 2024-03-16T10:11:39.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:39 smithi039 ceph-mon[31510]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 495 op/s 2024-03-16T10:11:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:39 smithi105 ceph-mon[36559]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 495 op/s 2024-03-16T10:11:41.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:41 smithi039 ceph-mon[31510]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 631 op/s 2024-03-16T10:11:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:41 smithi105 ceph-mon[36559]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 631 op/s 2024-03-16T10:11:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:41 smithi194 ceph-mon[36445]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 631 op/s 2024-03-16T10:11:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 120.0 s, 1323.8 tps, lat 75.574 ms stddev 103.434 2024-03-16T10:11:43.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:43 smithi039 ceph-mon[31510]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-16T10:11:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:43 smithi105 ceph-mon[36559]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-16T10:11:43.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:43 smithi194 ceph-mon[36445]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-16T10:11:45.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:45 smithi039 ceph-mon[31510]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-16T10:11:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:45 smithi105 ceph-mon[36559]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-16T10:11:45.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:45 smithi194 ceph-mon[36445]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 501 op/s 2024-03-16T10:11:47.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:47 smithi039 ceph-mon[31510]: pgmap v79: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 805 op/s 2024-03-16T10:11:47.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:47 smithi105 ceph-mon[36559]: pgmap v79: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 805 op/s 2024-03-16T10:11:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:47 smithi194 ceph-mon[36445]: pgmap v79: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.5 MiB/s wr, 805 op/s 2024-03-16T10:11:48.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 125.0 s, 1546.2 tps, lat 65.053 ms stddev 79.089 2024-03-16T10:11:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:49 smithi039 ceph-mon[31510]: pgmap v80: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 552 op/s 2024-03-16T10:11:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:49 smithi105 ceph-mon[36559]: pgmap v80: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 552 op/s 2024-03-16T10:11:49.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:49 smithi194 ceph-mon[36445]: pgmap v80: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 552 op/s 2024-03-16T10:11:51.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:51 smithi039 ceph-mon[31510]: pgmap v81: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-03-16T10:11:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:51 smithi105 ceph-mon[36559]: pgmap v81: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-03-16T10:11:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:51 smithi194 ceph-mon[36445]: pgmap v81: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-03-16T10:11:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 130.0 s, 1887.2 tps, lat 52.551 ms stddev 60.281 2024-03-16T10:11:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:53 smithi105 ceph-mon[36559]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-16T10:11:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:53 smithi194 ceph-mon[36445]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-16T10:11:53.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:53 smithi039 ceph-mon[31510]: pgmap v82: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 686 op/s 2024-03-16T10:11:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:55 smithi105 ceph-mon[36559]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-03-16T10:11:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:55 smithi194 ceph-mon[36445]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-03-16T10:11:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:55 smithi039 ceph-mon[31510]: pgmap v83: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-03-16T10:11:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:57 smithi105 ceph-mon[36559]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 828 op/s 2024-03-16T10:11:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:57 smithi194 ceph-mon[36445]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 828 op/s 2024-03-16T10:11:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:57 smithi039 ceph-mon[31510]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 9.1 MiB/s wr, 828 op/s 2024-03-16T10:11:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 135.0 s, 1208.4 tps, lat 83.028 ms stddev 101.176 2024-03-16T10:11:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:11:59 smithi105 ceph-mon[36559]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-03-16T10:11:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:11:59 smithi194 ceph-mon[36445]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-03-16T10:11:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:11:59 smithi039 ceph-mon[31510]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-03-16T10:12:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:01 smithi105 ceph-mon[36559]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-03-16T10:12:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:01 smithi194 ceph-mon[36445]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-03-16T10:12:01.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:01 smithi039 ceph-mon[31510]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.0 MiB/s wr, 641 op/s 2024-03-16T10:12:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 140.0 s, 611.4 tps, lat 158.436 ms stddev 202.382 2024-03-16T10:12:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:03 smithi105 ceph-mon[36559]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-03-16T10:12:03.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:03 smithi194 ceph-mon[36445]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-03-16T10:12:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:03 smithi039 ceph-mon[31510]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.7 MiB/s wr, 641 op/s 2024-03-16T10:12:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:05 smithi105 ceph-mon[36559]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-03-16T10:12:05.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:05 smithi194 ceph-mon[36445]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-03-16T10:12:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:05 smithi039 ceph-mon[31510]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-03-16T10:12:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:07 smithi105 ceph-mon[36559]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-03-16T10:12:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:07 smithi194 ceph-mon[36445]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-03-16T10:12:07.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:07 smithi039 ceph-mon[31510]: pgmap v89: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-03-16T10:12:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 145.0 s, 168.8 tps, lat 509.793 ms stddev 344.654 2024-03-16T10:12:09.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:09 smithi105 ceph-mon[36559]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-16T10:12:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:09 smithi194 ceph-mon[36445]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-16T10:12:09.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:09 smithi039 ceph-mon[31510]: pgmap v90: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.7 MiB/s wr, 376 op/s 2024-03-16T10:12:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:11 smithi105 ceph-mon[36559]: pgmap v91: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-03-16T10:12:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:11 smithi194 ceph-mon[36445]: pgmap v91: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-03-16T10:12:11.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:11 smithi039 ceph-mon[31510]: pgmap v91: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-03-16T10:12:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 150.0 s, 347.4 tps, lat 337.433 ms stddev 589.265 2024-03-16T10:12:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:13 smithi105 ceph-mon[36559]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-03-16T10:12:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:13 smithi194 ceph-mon[36445]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-03-16T10:12:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:13 smithi039 ceph-mon[31510]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2024-03-16T10:12:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:15 smithi105 ceph-mon[36559]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-16T10:12:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:15 smithi194 ceph-mon[36445]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-16T10:12:15.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:15 smithi039 ceph-mon[31510]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2024-03-16T10:12:16.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:16 smithi105 ceph-mon[36559]: pgmap v94: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.48k op/s 2024-03-16T10:12:16.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:16 smithi194 ceph-mon[36445]: pgmap v94: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.48k op/s 2024-03-16T10:12:16.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:16 smithi039 ceph-mon[31510]: pgmap v94: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 19 MiB/s wr, 1.48k op/s 2024-03-16T10:12:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 155.0 s, 1081.8 tps, lat 91.477 ms stddev 114.642 2024-03-16T10:12:19.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:19 smithi105 ceph-mon[36559]: pgmap v95: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.51k op/s 2024-03-16T10:12:19.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:19 smithi194 ceph-mon[36445]: pgmap v95: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.51k op/s 2024-03-16T10:12:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:19 smithi039 ceph-mon[31510]: pgmap v95: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 MiB/s wr, 1.51k op/s 2024-03-16T10:12:21.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:21 smithi105 ceph-mon[36559]: pgmap v96: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-03-16T10:12:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:21 smithi194 ceph-mon[36445]: pgmap v96: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-03-16T10:12:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:21 smithi039 ceph-mon[31510]: pgmap v96: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-03-16T10:12:23.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:23 smithi105 ceph-mon[36559]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-03-16T10:12:23.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:23 smithi194 ceph-mon[36445]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-03-16T10:12:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 160.0 s, 1092.2 tps, lat 91.522 ms stddev 104.343 2024-03-16T10:12:23.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:23 smithi039 ceph-mon[31510]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-03-16T10:12:25.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:25 smithi105 ceph-mon[36559]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-16T10:12:25.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:25 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:25 smithi194 ceph-mon[36445]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-16T10:12:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:25 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:25 smithi039 ceph-mon[31510]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-16T10:12:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:25 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:27.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:27 smithi105 ceph-mon[36559]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 948 op/s 2024-03-16T10:12:27.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:27 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:12:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:27 smithi194 ceph-mon[36445]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 948 op/s 2024-03-16T10:12:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:27 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:12:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:27 smithi039 ceph-mon[31510]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 948 op/s 2024-03-16T10:12:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:27 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:12:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 165.0 s, 1077.0 tps, lat 92.505 ms stddev 110.272 2024-03-16T10:12:29.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:29 smithi105 ceph-mon[36559]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 639 op/s 2024-03-16T10:12:29.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:29 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:12:29.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:29 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:12:29.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:29 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:12:29.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:29 smithi194 ceph-mon[36445]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 639 op/s 2024-03-16T10:12:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:29 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:12:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:29 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:12:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:29 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:12:29.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:29 smithi039 ceph-mon[31510]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 639 op/s 2024-03-16T10:12:29.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:29 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:12:29.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:29 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:12:29.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:29 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:12:31.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:31 smithi194 ceph-mon[36445]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2024-03-16T10:12:31.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:31 smithi039 ceph-mon[31510]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2024-03-16T10:12:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:31 smithi105 ceph-mon[36559]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2024-03-16T10:12:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:33 smithi194 ceph-mon[36445]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-16T10:12:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 170.0 s, 1180.0 tps, lat 85.451 ms stddev 105.945 2024-03-16T10:12:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:33 smithi039 ceph-mon[31510]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-16T10:12:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:33 smithi105 ceph-mon[36559]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-16T10:12:35.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:35 smithi194 ceph-mon[36445]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-16T10:12:35.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:35 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:35.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:35 smithi039 ceph-mon[31510]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-16T10:12:35.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:35 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:35 smithi105 ceph-mon[36559]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-03-16T10:12:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:35 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:37 smithi194 ceph-mon[36445]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-03-16T10:12:37.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:37 smithi039 ceph-mon[31510]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-03-16T10:12:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:37 smithi105 ceph-mon[36559]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-03-16T10:12:38.490 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 175.0 s, 1222.6 tps, lat 81.731 ms stddev 89.140 2024-03-16T10:12:39.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:39 smithi194 ceph-mon[36445]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-03-16T10:12:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:39 smithi039 ceph-mon[31510]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-03-16T10:12:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:39 smithi105 ceph-mon[36559]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 533 op/s 2024-03-16T10:12:41.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:41 smithi194 ceph-mon[36445]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 470 op/s 2024-03-16T10:12:41.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:41 smithi039 ceph-mon[31510]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 470 op/s 2024-03-16T10:12:41.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:41 smithi105 ceph-mon[36559]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.0 MiB/s wr, 470 op/s 2024-03-16T10:12:43.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:43 smithi194 ceph-mon[36445]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 772 op/s 2024-03-16T10:12:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 180.0 s, 917.8 tps, lat 108.688 ms stddev 117.562 2024-03-16T10:12:43.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:43 smithi039 ceph-mon[31510]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 772 op/s 2024-03-16T10:12:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:43 smithi105 ceph-mon[36559]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 772 op/s 2024-03-16T10:12:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:45 smithi039 ceph-mon[31510]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-03-16T10:12:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:45 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:45 smithi105 ceph-mon[36559]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-03-16T10:12:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:45 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:45.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:45 smithi194 ceph-mon[36445]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.8 MiB/s wr, 549 op/s 2024-03-16T10:12:45.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:45 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:12:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:47 smithi039 ceph-mon[31510]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 686 op/s 2024-03-16T10:12:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:47 smithi105 ceph-mon[36559]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 686 op/s 2024-03-16T10:12:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:47 smithi194 ceph-mon[36445]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.8 MiB/s wr, 686 op/s 2024-03-16T10:12:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 185.0 s, 831.8 tps, lat 119.442 ms stddev 131.133 2024-03-16T10:12:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:49 smithi039 ceph-mon[31510]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-16T10:12:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:49 smithi105 ceph-mon[36559]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-16T10:12:49.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:49 smithi194 ceph-mon[36445]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.0 MiB/s wr, 584 op/s 2024-03-16T10:12:51.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:51 smithi039 ceph-mon[31510]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-16T10:12:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:51 smithi105 ceph-mon[36559]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-16T10:12:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:51 smithi194 ceph-mon[36445]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-03-16T10:12:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 190.0 s, 997.4 tps, lat 99.181 ms stddev 128.389 2024-03-16T10:12:53.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:53 smithi039 ceph-mon[31510]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2024-03-16T10:12:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:53 smithi105 ceph-mon[36559]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2024-03-16T10:12:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:53 smithi194 ceph-mon[36445]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2024-03-16T10:12:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:55 smithi105 ceph-mon[36559]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-03-16T10:12:55.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:55 smithi194 ceph-mon[36445]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-03-16T10:12:55.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:55 smithi039 ceph-mon[31510]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-03-16T10:12:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:57 smithi105 ceph-mon[36559]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 637 op/s 2024-03-16T10:12:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:57 smithi194 ceph-mon[36445]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 637 op/s 2024-03-16T10:12:57.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:57 smithi039 ceph-mon[31510]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 637 op/s 2024-03-16T10:12:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 195.0 s, 1155.0 tps, lat 87.208 ms stddev 104.285 2024-03-16T10:12:59.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:12:59 smithi105 ceph-mon[36559]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 562 op/s 2024-03-16T10:12:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:12:59 smithi194 ceph-mon[36445]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 562 op/s 2024-03-16T10:12:59.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:12:59 smithi039 ceph-mon[31510]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.3 MiB/s wr, 562 op/s 2024-03-16T10:13:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:01 smithi105 ceph-mon[36559]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 479 op/s 2024-03-16T10:13:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:01 smithi194 ceph-mon[36445]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 479 op/s 2024-03-16T10:13:01.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:01 smithi039 ceph-mon[31510]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.5 MiB/s wr, 479 op/s 2024-03-16T10:13:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 200.0 s, 547.0 tps, lat 178.900 ms stddev 177.030 2024-03-16T10:13:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:03 smithi105 ceph-mon[36559]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 721 op/s 2024-03-16T10:13:03.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:03 smithi194 ceph-mon[36445]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 721 op/s 2024-03-16T10:13:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:03 smithi039 ceph-mon[31510]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.5 MiB/s wr, 721 op/s 2024-03-16T10:13:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:05 smithi105 ceph-mon[36559]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-03-16T10:13:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:05 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:05.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:05 smithi194 ceph-mon[36445]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-03-16T10:13:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:05 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:05 smithi039 ceph-mon[31510]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 501 op/s 2024-03-16T10:13:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:05 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:07 smithi105 ceph-mon[36559]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-03-16T10:13:07.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:07 smithi194 ceph-mon[36445]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-03-16T10:13:07.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:07 smithi039 ceph-mon[31510]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 1.20k op/s 2024-03-16T10:13:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 205.0 s, 273.2 tps, lat 371.327 ms stddev 587.817 2024-03-16T10:13:09.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:09 smithi105 ceph-mon[36559]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-03-16T10:13:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:09 smithi194 ceph-mon[36445]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-03-16T10:13:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:09 smithi039 ceph-mon[31510]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 1.40k op/s 2024-03-16T10:13:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:11 smithi105 ceph-mon[36559]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-03-16T10:13:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:11 smithi194 ceph-mon[36445]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-03-16T10:13:11.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:11 smithi039 ceph-mon[31510]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 1.35k op/s 2024-03-16T10:13:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 210.0 s, 742.6 tps, lat 129.697 ms stddev 162.568 2024-03-16T10:13:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:13 smithi105 ceph-mon[36559]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-03-16T10:13:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:13 smithi194 ceph-mon[36445]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-03-16T10:13:13.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:13 smithi039 ceph-mon[31510]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-03-16T10:13:15.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:15 smithi105 ceph-mon[36559]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-03-16T10:13:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:15 smithi194 ceph-mon[36445]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-03-16T10:13:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:15 smithi039 ceph-mon[31510]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.39k op/s 2024-03-16T10:13:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:17 smithi105 ceph-mon[36559]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-03-16T10:13:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:17 smithi194 ceph-mon[36445]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-03-16T10:13:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:17 smithi039 ceph-mon[31510]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-03-16T10:13:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 215.0 s, 882.2 tps, lat 118.985 ms stddev 140.381 2024-03-16T10:13:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:19 smithi105 ceph-mon[36559]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 917 op/s 2024-03-16T10:13:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:19 smithi194 ceph-mon[36445]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 917 op/s 2024-03-16T10:13:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:19 smithi039 ceph-mon[31510]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 917 op/s 2024-03-16T10:13:20.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:20 smithi105 ceph-mon[36559]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-16T10:13:20.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:20 smithi194 ceph-mon[36445]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-16T10:13:20.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:20 smithi039 ceph-mon[31510]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.7 MiB/s wr, 532 op/s 2024-03-16T10:13:23.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:23 smithi105 ceph-mon[36559]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 723 op/s 2024-03-16T10:13:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:23 smithi194 ceph-mon[36445]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 723 op/s 2024-03-16T10:13:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 220.0 s, 855.8 tps, lat 117.476 ms stddev 131.431 2024-03-16T10:13:23.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:23 smithi039 ceph-mon[31510]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.4 MiB/s wr, 723 op/s 2024-03-16T10:13:25.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:25 smithi105 ceph-mon[36559]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-03-16T10:13:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:25 smithi194 ceph-mon[36445]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-03-16T10:13:25.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:25 smithi039 ceph-mon[31510]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.1 MiB/s wr, 495 op/s 2024-03-16T10:13:27.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:27 smithi105 ceph-mon[36559]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 612 op/s 2024-03-16T10:13:27.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:27 smithi194 ceph-mon[36445]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 612 op/s 2024-03-16T10:13:27.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:27 smithi039 ceph-mon[31510]: pgmap v129: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 612 op/s 2024-03-16T10:13:28.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 225.0 s, 918.8 tps, lat 107.059 ms stddev 139.260 2024-03-16T10:13:29.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:29 smithi105 ceph-mon[36559]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 559 op/s 2024-03-16T10:13:29.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:29 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:13:29.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:29 smithi194 ceph-mon[36445]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 559 op/s 2024-03-16T10:13:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:29 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:13:29.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:29 smithi039 ceph-mon[31510]: pgmap v130: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 559 op/s 2024-03-16T10:13:29.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:29 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:13:31.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:31 smithi105 ceph-mon[36559]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-03-16T10:13:31.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:31 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:13:31.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:31 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:13:31.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:31 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:13:31.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:31 smithi194 ceph-mon[36445]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-03-16T10:13:31.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:31 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:13:31.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:31 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:13:31.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:31 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:13:31.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:31 smithi039 ceph-mon[31510]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.1 MiB/s wr, 484 op/s 2024-03-16T10:13:31.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:31 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:13:31.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:31 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:13:31.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:31 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:13:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:33 smithi194 ceph-mon[36445]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 673 op/s 2024-03-16T10:13:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 230.0 s, 787.8 tps, lat 128.325 ms stddev 138.770 2024-03-16T10:13:33.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:33 smithi039 ceph-mon[31510]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 673 op/s 2024-03-16T10:13:33.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:33 smithi105 ceph-mon[36559]: pgmap v132: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.8 MiB/s wr, 673 op/s 2024-03-16T10:13:35.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:35 smithi194 ceph-mon[36445]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-03-16T10:13:35.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:35 smithi039 ceph-mon[31510]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-03-16T10:13:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:35 smithi105 ceph-mon[36559]: pgmap v133: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-03-16T10:13:37.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:37 smithi194 ceph-mon[36445]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 608 op/s 2024-03-16T10:13:37.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:37 smithi039 ceph-mon[31510]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 608 op/s 2024-03-16T10:13:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:37 smithi105 ceph-mon[36559]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 608 op/s 2024-03-16T10:13:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 235.0 s, 992.2 tps, lat 96.600 ms stddev 110.668 2024-03-16T10:13:39.416 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:39 smithi105 ceph-mon[36559]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-03-16T10:13:39.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:39 smithi194 ceph-mon[36445]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-03-16T10:13:39.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:39 smithi039 ceph-mon[31510]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-03-16T10:13:40.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:40 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:40.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:40 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:40.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:40 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:41 smithi194 ceph-mon[36445]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:13:41.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:41 smithi039 ceph-mon[31510]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:13:41.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:41 smithi105 ceph-mon[36559]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:13:43.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:43 smithi194 ceph-mon[36445]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-03-16T10:13:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 240.0 s, 989.8 tps, lat 105.396 ms stddev 134.443 2024-03-16T10:13:43.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:43 smithi039 ceph-mon[31510]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-03-16T10:13:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:43 smithi105 ceph-mon[36559]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.0 MiB/s wr, 672 op/s 2024-03-16T10:13:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:45 smithi039 ceph-mon[31510]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 505 op/s 2024-03-16T10:13:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:45 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:45 smithi105 ceph-mon[36559]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 505 op/s 2024-03-16T10:13:45.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:45 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:45.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:45 smithi194 ceph-mon[36445]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 505 op/s 2024-03-16T10:13:45.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:45 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:13:47.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:47 smithi039 ceph-mon[31510]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.2 MiB/s wr, 698 op/s 2024-03-16T10:13:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:47 smithi105 ceph-mon[36559]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.2 MiB/s wr, 698 op/s 2024-03-16T10:13:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:47 smithi194 ceph-mon[36445]: pgmap v139: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.2 MiB/s wr, 698 op/s 2024-03-16T10:13:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 245.0 s, 710.4 tps, lat 140.638 ms stddev 159.915 2024-03-16T10:13:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:49 smithi039 ceph-mon[31510]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-16T10:13:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:49 smithi105 ceph-mon[36559]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-16T10:13:49.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:49 smithi194 ceph-mon[36445]: pgmap v140: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-16T10:13:51.508 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:51 smithi039 ceph-mon[31510]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-16T10:13:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:51 smithi105 ceph-mon[36559]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-16T10:13:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:51 smithi194 ceph-mon[36445]: pgmap v141: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-16T10:13:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 250.0 s, 597.8 tps, lat 161.158 ms stddev 187.020 2024-03-16T10:13:53.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:53 smithi039 ceph-mon[31510]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-03-16T10:13:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:53 smithi105 ceph-mon[36559]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-03-16T10:13:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:53 smithi194 ceph-mon[36445]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.1 MiB/s wr, 751 op/s 2024-03-16T10:13:55.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:55 smithi039 ceph-mon[31510]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 MiB/s wr, 558 op/s 2024-03-16T10:13:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:55 smithi105 ceph-mon[36559]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 MiB/s wr, 558 op/s 2024-03-16T10:13:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:55 smithi194 ceph-mon[36445]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.7 MiB/s wr, 558 op/s 2024-03-16T10:13:57.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:57 smithi105 ceph-mon[36559]: pgmap v144: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-03-16T10:13:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:57 smithi194 ceph-mon[36445]: pgmap v144: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-03-16T10:13:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:57 smithi039 ceph-mon[31510]: pgmap v144: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 666 op/s 2024-03-16T10:13:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 255.0 s, 607.0 tps, lat 169.377 ms stddev 206.327 2024-03-16T10:13:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:13:59 smithi105 ceph-mon[36559]: pgmap v145: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 541 op/s 2024-03-16T10:13:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:13:59 smithi194 ceph-mon[36445]: pgmap v145: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 541 op/s 2024-03-16T10:13:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:13:59 smithi039 ceph-mon[31510]: pgmap v145: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 541 op/s 2024-03-16T10:14:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:01 smithi105 ceph-mon[36559]: pgmap v146: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-03-16T10:14:01.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:01 smithi194 ceph-mon[36445]: pgmap v146: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-03-16T10:14:01.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:01 smithi039 ceph-mon[31510]: pgmap v146: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-03-16T10:14:03.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 260.0 s, 396.6 tps, lat 247.340 ms stddev 372.495 2024-03-16T10:14:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:03 smithi105 ceph-mon[36559]: pgmap v147: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 636 op/s 2024-03-16T10:14:03.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:03 smithi194 ceph-mon[36445]: pgmap v147: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 636 op/s 2024-03-16T10:14:03.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:03 smithi039 ceph-mon[31510]: pgmap v147: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.2 MiB/s wr, 636 op/s 2024-03-16T10:14:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:05 smithi105 ceph-mon[36559]: pgmap v148: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.1 MiB/s wr, 506 op/s 2024-03-16T10:14:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:05 smithi194 ceph-mon[36445]: pgmap v148: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.1 MiB/s wr, 506 op/s 2024-03-16T10:14:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:05 smithi039 ceph-mon[31510]: pgmap v148: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.1 MiB/s wr, 506 op/s 2024-03-16T10:14:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:07 smithi105 ceph-mon[36559]: pgmap v149: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 967 op/s 2024-03-16T10:14:07.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:07 smithi194 ceph-mon[36445]: pgmap v149: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 967 op/s 2024-03-16T10:14:07.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:07 smithi039 ceph-mon[31510]: pgmap v149: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 967 op/s 2024-03-16T10:14:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 265.0 s, 548.4 tps, lat 183.738 ms stddev 366.878 2024-03-16T10:14:09.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:09 smithi105 ceph-mon[36559]: pgmap v150: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 995 op/s 2024-03-16T10:14:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:09 smithi194 ceph-mon[36445]: pgmap v150: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 995 op/s 2024-03-16T10:14:09.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:09 smithi039 ceph-mon[31510]: pgmap v150: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.9 MiB/s wr, 995 op/s 2024-03-16T10:14:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:11 smithi105 ceph-mon[36559]: pgmap v151: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 982 op/s 2024-03-16T10:14:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:11 smithi194 ceph-mon[36445]: pgmap v151: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 982 op/s 2024-03-16T10:14:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:11 smithi039 ceph-mon[31510]: pgmap v151: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.7 MiB/s wr, 982 op/s 2024-03-16T10:14:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 270.0 s, 660.2 tps, lat 153.796 ms stddev 182.395 2024-03-16T10:14:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:13 smithi105 ceph-mon[36559]: pgmap v152: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-03-16T10:14:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:13 smithi194 ceph-mon[36445]: pgmap v152: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-03-16T10:14:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:13 smithi039 ceph-mon[31510]: pgmap v152: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-03-16T10:14:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:15 smithi105 ceph-mon[36559]: pgmap v153: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2024-03-16T10:14:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:15 smithi194 ceph-mon[36445]: pgmap v153: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2024-03-16T10:14:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:15 smithi039 ceph-mon[31510]: pgmap v153: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2024-03-16T10:14:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:17 smithi105 ceph-mon[36559]: pgmap v154: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-03-16T10:14:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:17 smithi194 ceph-mon[36445]: pgmap v154: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-03-16T10:14:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:17 smithi039 ceph-mon[31510]: pgmap v154: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 1.15k op/s 2024-03-16T10:14:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 275.0 s, 601.0 tps, lat 163.971 ms stddev 182.943 2024-03-16T10:14:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:19 smithi105 ceph-mon[36559]: pgmap v155: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 714 op/s 2024-03-16T10:14:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:19 smithi194 ceph-mon[36445]: pgmap v155: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 714 op/s 2024-03-16T10:14:19.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:19 smithi039 ceph-mon[31510]: pgmap v155: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 714 op/s 2024-03-16T10:14:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:21 smithi194 ceph-mon[36445]: pgmap v156: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-03-16T10:14:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:21 smithi039 ceph-mon[31510]: pgmap v156: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-03-16T10:14:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:21 smithi105 ceph-mon[36559]: pgmap v156: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-03-16T10:14:22.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:22 smithi194 ceph-mon[36445]: pgmap v157: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-03-16T10:14:22.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:22 smithi039 ceph-mon[31510]: pgmap v157: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-03-16T10:14:22.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:22 smithi105 ceph-mon[36559]: pgmap v157: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-03-16T10:14:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 280.0 s, 535.0 tps, lat 189.774 ms stddev 231.809 2024-03-16T10:14:25.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:25 smithi105 ceph-mon[36559]: pgmap v158: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 531 op/s 2024-03-16T10:14:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:25 smithi194 ceph-mon[36445]: pgmap v158: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 531 op/s 2024-03-16T10:14:25.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:25 smithi039 ceph-mon[31510]: pgmap v158: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 531 op/s 2024-03-16T10:14:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:27 smithi194 ceph-mon[36445]: pgmap v159: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 629 op/s 2024-03-16T10:14:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:27 smithi039 ceph-mon[31510]: pgmap v159: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 629 op/s 2024-03-16T10:14:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:27 smithi105 ceph-mon[36559]: pgmap v159: 129 pgs: 129 active+clean; 2.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 629 op/s 2024-03-16T10:14:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 285.0 s, 575.0 tps, lat 172.228 ms stddev 177.527 2024-03-16T10:14:29.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:29 smithi194 ceph-mon[36445]: pgmap v160: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-03-16T10:14:29.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:29 smithi039 ceph-mon[31510]: pgmap v160: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-03-16T10:14:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:29 smithi105 ceph-mon[36559]: pgmap v160: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 481 op/s 2024-03-16T10:14:31.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:31 smithi194 ceph-mon[36445]: pgmap v161: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 472 op/s 2024-03-16T10:14:31.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:31 smithi039 ceph-mon[31510]: pgmap v161: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 472 op/s 2024-03-16T10:14:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:31 smithi105 ceph-mon[36559]: pgmap v161: 129 pgs: 129 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.8 MiB/s wr, 472 op/s 2024-03-16T10:14:32.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:32 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:14:32.508 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:32 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:14:32.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:32 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:14:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:33 smithi194 ceph-mon[36445]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-03-16T10:14:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:33 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:14:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:33 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:14:33.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:33 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:14:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 290.0 s, 549.8 tps, lat 174.744 ms stddev 211.221 2024-03-16T10:14:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:33 smithi039 ceph-mon[31510]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-03-16T10:14:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:33 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:14:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:33 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:14:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:33 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:14:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:33 smithi105 ceph-mon[36559]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-03-16T10:14:33.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:33 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:14:33.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:33 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:14:33.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:33 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:14:35.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:35 smithi194 ceph-mon[36445]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-03-16T10:14:35.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:35 smithi039 ceph-mon[31510]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-03-16T10:14:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:35 smithi105 ceph-mon[36559]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 486 op/s 2024-03-16T10:14:37.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:37 smithi194 ceph-mon[36445]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-03-16T10:14:37.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:37 smithi039 ceph-mon[31510]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-03-16T10:14:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:37 smithi105 ceph-mon[36559]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.9 MiB/s wr, 604 op/s 2024-03-16T10:14:38.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 295.0 s, 523.3 tps, lat 196.222 ms stddev 245.280 2024-03-16T10:14:39.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:39 smithi194 ceph-mon[36445]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 506 op/s 2024-03-16T10:14:39.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:39 smithi039 ceph-mon[31510]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 506 op/s 2024-03-16T10:14:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:39 smithi105 ceph-mon[36559]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 506 op/s 2024-03-16T10:14:41.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:41 smithi194 ceph-mon[36445]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 501 op/s 2024-03-16T10:14:41.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:41 smithi039 ceph-mon[31510]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 501 op/s 2024-03-16T10:14:41.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:41 smithi105 ceph-mon[36559]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.8 MiB/s wr, 501 op/s 2024-03-16T10:14:43.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:43 smithi194 ceph-mon[36445]: pgmap v167: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 633 op/s 2024-03-16T10:14:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 300.0 s, 584.3 tps, lat 174.548 ms stddev 210.329 2024-03-16T10:14:43.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:43 smithi039 ceph-mon[31510]: pgmap v167: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 633 op/s 2024-03-16T10:14:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:43 smithi105 ceph-mon[36559]: pgmap v167: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 633 op/s 2024-03-16T10:14:45.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:45 smithi039 ceph-mon[31510]: pgmap v168: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-03-16T10:14:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:45 smithi105 ceph-mon[36559]: pgmap v168: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-03-16T10:14:45.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:45 smithi194 ceph-mon[36445]: pgmap v168: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 540 op/s 2024-03-16T10:14:47.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:47 smithi039 ceph-mon[31510]: pgmap v169: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 709 op/s 2024-03-16T10:14:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:47 smithi105 ceph-mon[36559]: pgmap v169: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 709 op/s 2024-03-16T10:14:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:47 smithi194 ceph-mon[36445]: pgmap v169: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 16 MiB/s wr, 709 op/s 2024-03-16T10:14:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 305.0 s, 575.2 tps, lat 166.898 ms stddev 197.825 2024-03-16T10:14:49.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:49 smithi039 ceph-mon[31510]: pgmap v170: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 606 op/s 2024-03-16T10:14:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:49 smithi105 ceph-mon[36559]: pgmap v170: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 606 op/s 2024-03-16T10:14:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:49 smithi194 ceph-mon[36445]: pgmap v170: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 13 MiB/s wr, 606 op/s 2024-03-16T10:14:51.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:51 smithi039 ceph-mon[31510]: pgmap v171: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 585 op/s 2024-03-16T10:14:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:51 smithi105 ceph-mon[36559]: pgmap v171: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 585 op/s 2024-03-16T10:14:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:51 smithi194 ceph-mon[36445]: pgmap v171: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 585 op/s 2024-03-16T10:14:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 310.0 s, 632.4 tps, lat 160.491 ms stddev 204.834 2024-03-16T10:14:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:53 smithi105 ceph-mon[36559]: pgmap v172: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 718 op/s 2024-03-16T10:14:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:53 smithi194 ceph-mon[36445]: pgmap v172: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 718 op/s 2024-03-16T10:14:53.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:53 smithi039 ceph-mon[31510]: pgmap v172: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 718 op/s 2024-03-16T10:14:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:55 smithi105 ceph-mon[36559]: pgmap v173: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-03-16T10:14:55.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:55 smithi194 ceph-mon[36445]: pgmap v173: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-03-16T10:14:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:55 smithi039 ceph-mon[31510]: pgmap v173: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 599 op/s 2024-03-16T10:14:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:57 smithi105 ceph-mon[36559]: pgmap v174: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 670 op/s 2024-03-16T10:14:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:57 smithi194 ceph-mon[36445]: pgmap v174: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 670 op/s 2024-03-16T10:14:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:57 smithi039 ceph-mon[31510]: pgmap v174: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 670 op/s 2024-03-16T10:14:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 315.0 s, 591.0 tps, lat 168.251 ms stddev 219.338 2024-03-16T10:14:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:14:59 smithi105 ceph-mon[36559]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-16T10:14:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:14:59 smithi194 ceph-mon[36445]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-16T10:14:59.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:14:59 smithi039 ceph-mon[31510]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 505 op/s 2024-03-16T10:15:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:01 smithi105 ceph-mon[36559]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 483 op/s 2024-03-16T10:15:01.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:01 smithi194 ceph-mon[36445]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 483 op/s 2024-03-16T10:15:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:01 smithi039 ceph-mon[31510]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 483 op/s 2024-03-16T10:15:02.617 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:02 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:02.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:02 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:02.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:02 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 320.0 s, 683.0 tps, lat 151.959 ms stddev 238.960 2024-03-16T10:15:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:03 smithi105 ceph-mon[36559]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 638 op/s 2024-03-16T10:15:03.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:03 smithi194 ceph-mon[36445]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 638 op/s 2024-03-16T10:15:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:03 smithi039 ceph-mon[31510]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 638 op/s 2024-03-16T10:15:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:05 smithi105 ceph-mon[36559]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 540 op/s 2024-03-16T10:15:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:05 smithi194 ceph-mon[36445]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 540 op/s 2024-03-16T10:15:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:05 smithi039 ceph-mon[31510]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 540 op/s 2024-03-16T10:15:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:07 smithi105 ceph-mon[36559]: pgmap v179: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 686 op/s 2024-03-16T10:15:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:07 smithi194 ceph-mon[36445]: pgmap v179: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 686 op/s 2024-03-16T10:15:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:07 smithi039 ceph-mon[31510]: pgmap v179: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 686 op/s 2024-03-16T10:15:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 325.0 s, 536.6 tps, lat 180.603 ms stddev 225.667 2024-03-16T10:15:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:09 smithi194 ceph-mon[36445]: pgmap v180: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 591 op/s 2024-03-16T10:15:09.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:09 smithi039 ceph-mon[31510]: pgmap v180: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 591 op/s 2024-03-16T10:15:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:09 smithi105 ceph-mon[36559]: pgmap v180: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 12 MiB/s wr, 591 op/s 2024-03-16T10:15:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:11 smithi194 ceph-mon[36445]: pgmap v181: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-16T10:15:11.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:11 smithi039 ceph-mon[31510]: pgmap v181: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-16T10:15:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:11 smithi105 ceph-mon[36559]: pgmap v181: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 581 op/s 2024-03-16T10:15:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 330.0 s, 558.6 tps, lat 181.874 ms stddev 220.046 2024-03-16T10:15:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:13 smithi194 ceph-mon[36445]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 706 op/s 2024-03-16T10:15:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:13 smithi039 ceph-mon[31510]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 706 op/s 2024-03-16T10:15:13.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:13 smithi105 ceph-mon[36559]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 706 op/s 2024-03-16T10:15:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:15 smithi194 ceph-mon[36445]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 576 op/s 2024-03-16T10:15:15.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:15 smithi039 ceph-mon[31510]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 576 op/s 2024-03-16T10:15:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:15 smithi105 ceph-mon[36559]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 576 op/s 2024-03-16T10:15:16.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:16 smithi194 ceph-mon[36445]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-03-16T10:15:16.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:16 smithi039 ceph-mon[31510]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-03-16T10:15:16.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:16 smithi105 ceph-mon[36559]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 678 op/s 2024-03-16T10:15:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 335.0 s, 638.4 tps, lat 158.204 ms stddev 186.720 2024-03-16T10:15:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:19 smithi194 ceph-mon[36445]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-03-16T10:15:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:19 smithi039 ceph-mon[31510]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-03-16T10:15:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:19 smithi105 ceph-mon[36559]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 528 op/s 2024-03-16T10:15:21.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:21 smithi194 ceph-mon[36445]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-03-16T10:15:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:21 smithi039 ceph-mon[31510]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-03-16T10:15:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:21 smithi105 ceph-mon[36559]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.6 MiB/s wr, 516 op/s 2024-03-16T10:15:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:23 smithi194 ceph-mon[36445]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 652 op/s 2024-03-16T10:15:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 340.0 s, 635.2 tps, lat 154.279 ms stddev 177.274 2024-03-16T10:15:23.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:23 smithi039 ceph-mon[31510]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 652 op/s 2024-03-16T10:15:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:23 smithi105 ceph-mon[36559]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 652 op/s 2024-03-16T10:15:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:25 smithi194 ceph-mon[36445]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 538 op/s 2024-03-16T10:15:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:25 smithi105 ceph-mon[36559]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 538 op/s 2024-03-16T10:15:25.716 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:25 smithi039 ceph-mon[31510]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 538 op/s 2024-03-16T10:15:27.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:27 smithi194 ceph-mon[36445]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-16T10:15:27.514 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:27 smithi039 ceph-mon[31510]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-16T10:15:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:27 smithi105 ceph-mon[36559]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 619 op/s 2024-03-16T10:15:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 345.0 s, 670.4 tps, lat 144.882 ms stddev 165.399 2024-03-16T10:15:29.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:29 smithi194 ceph-mon[36445]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.2 MiB/s wr, 500 op/s 2024-03-16T10:15:29.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:29 smithi039 ceph-mon[31510]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.2 MiB/s wr, 500 op/s 2024-03-16T10:15:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:29 smithi105 ceph-mon[36559]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.2 MiB/s wr, 500 op/s 2024-03-16T10:15:30.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:30 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:30.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:30 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:30.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:30 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:15:31.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:31 smithi039 ceph-mon[31510]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 486 op/s 2024-03-16T10:15:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:31 smithi105 ceph-mon[36559]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 486 op/s 2024-03-16T10:15:31.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:31 smithi194 ceph-mon[36445]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 486 op/s 2024-03-16T10:15:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 350.0 s, 482.8 tps, lat 213.334 ms stddev 313.176 2024-03-16T10:15:33.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:33 smithi039 ceph-mon[31510]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-16T10:15:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:33 smithi105 ceph-mon[36559]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-16T10:15:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:33 smithi194 ceph-mon[36445]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-16T10:15:34.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:34 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:15:34.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:34 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:15:34.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:34 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:15:35.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:35 smithi039 ceph-mon[31510]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.1 MiB/s wr, 486 op/s 2024-03-16T10:15:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:35 smithi105 ceph-mon[36559]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.1 MiB/s wr, 486 op/s 2024-03-16T10:15:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:35 smithi194 ceph-mon[36445]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.1 MiB/s wr, 486 op/s 2024-03-16T10:15:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:36 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:15:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:36 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:15:36.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:36 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:15:36.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:36 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:15:36.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:36 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:15:36.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:36 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:15:36.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:36 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:15:36.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:36 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:15:36.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:36 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:15:37.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:37 smithi039 ceph-mon[31510]: pgmap v194: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 567 op/s 2024-03-16T10:15:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:37 smithi105 ceph-mon[36559]: pgmap v194: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 567 op/s 2024-03-16T10:15:37.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:37 smithi194 ceph-mon[36445]: pgmap v194: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.8 MiB/s wr, 567 op/s 2024-03-16T10:15:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 355.0 s, 430.4 tps, lat 226.785 ms stddev 275.926 2024-03-16T10:15:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:39 smithi105 ceph-mon[36559]: pgmap v195: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 490 op/s 2024-03-16T10:15:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:39 smithi194 ceph-mon[36445]: pgmap v195: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 490 op/s 2024-03-16T10:15:39.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:39 smithi039 ceph-mon[31510]: pgmap v195: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 490 op/s 2024-03-16T10:15:41.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:41 smithi105 ceph-mon[36559]: pgmap v196: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 484 op/s 2024-03-16T10:15:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:41 smithi194 ceph-mon[36445]: pgmap v196: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 484 op/s 2024-03-16T10:15:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:41 smithi039 ceph-mon[31510]: pgmap v196: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.5 MiB/s wr, 484 op/s 2024-03-16T10:15:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 360.0 s, 536.8 tps, lat 189.611 ms stddev 246.459 2024-03-16T10:15:43.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:43 smithi039 ceph-mon[31510]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-16T10:15:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:43 smithi105 ceph-mon[36559]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-16T10:15:43.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:43 smithi194 ceph-mon[36445]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 649 op/s 2024-03-16T10:15:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:45 smithi105 ceph-mon[36559]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 566 op/s 2024-03-16T10:15:45.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:45 smithi194 ceph-mon[36445]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 566 op/s 2024-03-16T10:15:45.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:45 smithi039 ceph-mon[31510]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.2 MiB/s wr, 566 op/s 2024-03-16T10:15:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:47 smithi105 ceph-mon[36559]: pgmap v199: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-16T10:15:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:47 smithi194 ceph-mon[36445]: pgmap v199: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-16T10:15:47.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:47 smithi039 ceph-mon[31510]: pgmap v199: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-16T10:15:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 365.0 s, 555.4 tps, lat 181.080 ms stddev 245.461 2024-03-16T10:15:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:49 smithi105 ceph-mon[36559]: pgmap v200: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-03-16T10:15:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:49 smithi194 ceph-mon[36445]: pgmap v200: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-03-16T10:15:49.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:49 smithi039 ceph-mon[31510]: pgmap v200: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.6 MiB/s wr, 581 op/s 2024-03-16T10:15:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:51 smithi105 ceph-mon[36559]: pgmap v201: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 554 op/s 2024-03-16T10:15:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:51 smithi194 ceph-mon[36445]: pgmap v201: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 554 op/s 2024-03-16T10:15:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:51 smithi039 ceph-mon[31510]: pgmap v201: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 554 op/s 2024-03-16T10:15:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 370.0 s, 530.2 tps, lat 182.395 ms stddev 239.216 2024-03-16T10:15:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:53 smithi105 ceph-mon[36559]: pgmap v202: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 647 op/s 2024-03-16T10:15:53.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:53 smithi194 ceph-mon[36445]: pgmap v202: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 647 op/s 2024-03-16T10:15:53.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:53 smithi039 ceph-mon[31510]: pgmap v202: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 647 op/s 2024-03-16T10:15:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:55 smithi105 ceph-mon[36559]: pgmap v203: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 519 op/s 2024-03-16T10:15:55.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:55 smithi194 ceph-mon[36445]: pgmap v203: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 519 op/s 2024-03-16T10:15:55.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:55 smithi039 ceph-mon[31510]: pgmap v203: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 519 op/s 2024-03-16T10:15:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:57 smithi105 ceph-mon[36559]: pgmap v204: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 581 op/s 2024-03-16T10:15:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:57 smithi194 ceph-mon[36445]: pgmap v204: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 581 op/s 2024-03-16T10:15:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:57 smithi039 ceph-mon[31510]: pgmap v204: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.6 MiB/s wr, 581 op/s 2024-03-16T10:15:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 375.0 s, 509.2 tps, lat 203.704 ms stddev 257.465 2024-03-16T10:15:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:15:59 smithi105 ceph-mon[36559]: pgmap v205: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 478 op/s 2024-03-16T10:15:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:15:59 smithi194 ceph-mon[36445]: pgmap v205: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 478 op/s 2024-03-16T10:15:59.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:15:59 smithi039 ceph-mon[31510]: pgmap v205: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.0 MiB/s wr, 478 op/s 2024-03-16T10:16:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:01 smithi105 ceph-mon[36559]: pgmap v206: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-16T10:16:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:01 smithi194 ceph-mon[36445]: pgmap v206: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-16T10:16:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:01 smithi039 ceph-mon[31510]: pgmap v206: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-16T10:16:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 380.0 s, 250.8 tps, lat 383.190 ms stddev 714.383 2024-03-16T10:16:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:03 smithi105 ceph-mon[36559]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-16T10:16:03.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:03 smithi194 ceph-mon[36445]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-16T10:16:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:03 smithi039 ceph-mon[31510]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 731 op/s 2024-03-16T10:16:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:05 smithi105 ceph-mon[36559]: pgmap v208: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 983 op/s 2024-03-16T10:16:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:05 smithi105 ceph-mon[36559]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T10:16:05.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:05 smithi194 ceph-mon[36445]: pgmap v208: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 983 op/s 2024-03-16T10:16:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:05 smithi194 ceph-mon[36445]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T10:16:05.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:05 smithi039 ceph-mon[31510]: pgmap v208: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 983 op/s 2024-03-16T10:16:05.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:05 smithi039 ceph-mon[31510]: osdmap e96: 12 total, 12 up, 12 in 2024-03-16T10:16:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:07 smithi105 ceph-mon[36559]: pgmap v210: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-03-16T10:16:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:07 smithi194 ceph-mon[36445]: pgmap v210: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-03-16T10:16:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:07 smithi039 ceph-mon[31510]: pgmap v210: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-03-16T10:16:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 385.0 s, 306.8 tps, lat 302.922 ms stddev 438.761 2024-03-16T10:16:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:09 smithi194 ceph-mon[36445]: pgmap v211: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.62k op/s 2024-03-16T10:16:09.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:09 smithi039 ceph-mon[31510]: pgmap v211: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.62k op/s 2024-03-16T10:16:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:09 smithi105 ceph-mon[36559]: pgmap v211: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 1.62k op/s 2024-03-16T10:16:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:11 smithi194 ceph-mon[36445]: pgmap v212: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-03-16T10:16:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:11 smithi039 ceph-mon[31510]: pgmap v212: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-03-16T10:16:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:11 smithi105 ceph-mon[36559]: pgmap v212: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 1.67k op/s 2024-03-16T10:16:13.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:13 smithi105 ceph-mon[36559]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T10:16:13.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:13 smithi105 ceph-mon[36559]: pgmap v214: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-03-16T10:16:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:13 smithi194 ceph-mon[36445]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T10:16:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:13 smithi194 ceph-mon[36445]: pgmap v214: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-03-16T10:16:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 390.0 s, 488.2 tps, lat 227.698 ms stddev 313.053 2024-03-16T10:16:13.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:13 smithi039 ceph-mon[31510]: osdmap e97: 12 total, 12 up, 12 in 2024-03-16T10:16:13.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:13 smithi039 ceph-mon[31510]: pgmap v214: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-03-16T10:16:15.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:15 smithi194 ceph-mon[36445]: pgmap v215: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-16T10:16:15.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:15 smithi039 ceph-mon[31510]: pgmap v215: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-16T10:16:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:15 smithi105 ceph-mon[36559]: pgmap v215: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 MiB/s wr, 1.09k op/s 2024-03-16T10:16:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:17 smithi194 ceph-mon[36445]: pgmap v216: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-16T10:16:17.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:17 smithi194 ceph-mon[36445]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T10:16:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:17 smithi039 ceph-mon[31510]: pgmap v216: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-16T10:16:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:17 smithi039 ceph-mon[31510]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T10:16:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:17 smithi105 ceph-mon[36559]: pgmap v216: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-16T10:16:17.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:17 smithi105 ceph-mon[36559]: osdmap e98: 12 total, 12 up, 12 in 2024-03-16T10:16:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 395.0 s, 542.0 tps, lat 186.072 ms stddev 210.462 2024-03-16T10:16:19.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:19 smithi194 ceph-mon[36445]: pgmap v218: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-03-16T10:16:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:19 smithi039 ceph-mon[31510]: pgmap v218: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-03-16T10:16:19.618 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:19 smithi105 ceph-mon[36559]: pgmap v218: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-03-16T10:16:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:21 smithi039 ceph-mon[31510]: pgmap v219: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 430 op/s 2024-03-16T10:16:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:21 smithi105 ceph-mon[36559]: pgmap v219: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 430 op/s 2024-03-16T10:16:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:21 smithi194 ceph-mon[36445]: pgmap v219: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 430 op/s 2024-03-16T10:16:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 400.0 s, 511.6 tps, lat 194.382 ms stddev 241.469 2024-03-16T10:16:23.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:23 smithi039 ceph-mon[31510]: pgmap v220: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 525 op/s 2024-03-16T10:16:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:23 smithi105 ceph-mon[36559]: pgmap v220: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 525 op/s 2024-03-16T10:16:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:23 smithi194 ceph-mon[36445]: pgmap v220: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.6 MiB/s wr, 525 op/s 2024-03-16T10:16:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:25 smithi039 ceph-mon[31510]: pgmap v221: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-03-16T10:16:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:25 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:25 smithi105 ceph-mon[36559]: pgmap v221: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-03-16T10:16:25.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:25 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:25 smithi194 ceph-mon[36445]: pgmap v221: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.7 MiB/s wr, 503 op/s 2024-03-16T10:16:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:25 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:27.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:27 smithi039 ceph-mon[31510]: pgmap v222: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 504 op/s 2024-03-16T10:16:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:27 smithi105 ceph-mon[36559]: pgmap v222: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 504 op/s 2024-03-16T10:16:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:27 smithi194 ceph-mon[36445]: pgmap v222: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 504 op/s 2024-03-16T10:16:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 405.0 s, 505.0 tps, lat 192.514 ms stddev 253.776 2024-03-16T10:16:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:29 smithi105 ceph-mon[36559]: pgmap v223: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 480 op/s 2024-03-16T10:16:29.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:29 smithi194 ceph-mon[36445]: pgmap v223: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 480 op/s 2024-03-16T10:16:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:29 smithi039 ceph-mon[31510]: pgmap v223: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 480 op/s 2024-03-16T10:16:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:31 smithi105 ceph-mon[36559]: pgmap v224: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-03-16T10:16:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:31 smithi194 ceph-mon[36445]: pgmap v224: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-03-16T10:16:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:31 smithi039 ceph-mon[31510]: pgmap v224: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-03-16T10:16:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 410.0 s, 564.8 tps, lat 181.228 ms stddev 201.535 2024-03-16T10:16:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:33 smithi105 ceph-mon[36559]: pgmap v225: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 573 op/s 2024-03-16T10:16:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:33 smithi194 ceph-mon[36445]: pgmap v225: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 573 op/s 2024-03-16T10:16:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:33 smithi039 ceph-mon[31510]: pgmap v225: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 573 op/s 2024-03-16T10:16:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:35 smithi105 ceph-mon[36559]: pgmap v226: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 471 op/s 2024-03-16T10:16:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:35 smithi194 ceph-mon[36445]: pgmap v226: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 471 op/s 2024-03-16T10:16:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:35 smithi039 ceph-mon[31510]: pgmap v226: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 471 op/s 2024-03-16T10:16:36.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:36 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:16:36.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:36 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:16:36.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:36 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:16:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:37 smithi105 ceph-mon[36559]: pgmap v227: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2024-03-16T10:16:37.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:37 smithi194 ceph-mon[36445]: pgmap v227: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2024-03-16T10:16:37.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:37 smithi039 ceph-mon[31510]: pgmap v227: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2024-03-16T10:16:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 415.0 s, 435.8 tps, lat 227.339 ms stddev 264.593 2024-03-16T10:16:38.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:38 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:16:38.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:38 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:16:38.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:38 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:16:38.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:38 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:16:38.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:38 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:16:38.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:38 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:16:38.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:38 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:16:38.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:38 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:16:38.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:38 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:16:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:39 smithi105 ceph-mon[36559]: pgmap v228: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 546 op/s 2024-03-16T10:16:39.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:39 smithi194 ceph-mon[36445]: pgmap v228: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 546 op/s 2024-03-16T10:16:39.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:39 smithi039 ceph-mon[31510]: pgmap v228: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 546 op/s 2024-03-16T10:16:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:41 smithi105 ceph-mon[36559]: pgmap v229: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-03-16T10:16:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:41 smithi194 ceph-mon[36445]: pgmap v229: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-03-16T10:16:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:41 smithi039 ceph-mon[31510]: pgmap v229: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.8 MiB/s wr, 489 op/s 2024-03-16T10:16:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 420.0 s, 431.0 tps, lat 230.139 ms stddev 264.266 2024-03-16T10:16:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:43 smithi105 ceph-mon[36559]: pgmap v230: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-03-16T10:16:43.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:43 smithi194 ceph-mon[36445]: pgmap v230: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-03-16T10:16:43.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:43 smithi039 ceph-mon[31510]: pgmap v230: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.2 MiB/s wr, 585 op/s 2024-03-16T10:16:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:45 smithi105 ceph-mon[36559]: pgmap v231: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.9 MiB/s wr, 474 op/s 2024-03-16T10:16:45.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:45 smithi194 ceph-mon[36445]: pgmap v231: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.9 MiB/s wr, 474 op/s 2024-03-16T10:16:45.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:45 smithi039 ceph-mon[31510]: pgmap v231: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.9 MiB/s wr, 474 op/s 2024-03-16T10:16:47.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:47 smithi105 ceph-mon[36559]: pgmap v232: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.2 MiB/s wr, 496 op/s 2024-03-16T10:16:47.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:47 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:47 smithi194 ceph-mon[36445]: pgmap v232: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.2 MiB/s wr, 496 op/s 2024-03-16T10:16:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:47 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:47 smithi039 ceph-mon[31510]: pgmap v232: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 6.2 MiB/s wr, 496 op/s 2024-03-16T10:16:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:47 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:16:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 425.0 s, 470.2 tps, lat 215.255 ms stddev 238.666 2024-03-16T10:16:49.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:49 smithi105 ceph-mon[36559]: pgmap v233: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2024-03-16T10:16:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:49 smithi194 ceph-mon[36445]: pgmap v233: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2024-03-16T10:16:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:49 smithi039 ceph-mon[31510]: pgmap v233: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 7.0 MiB/s wr, 545 op/s 2024-03-16T10:16:51.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:51 smithi105 ceph-mon[36559]: pgmap v234: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-03-16T10:16:51.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:51 smithi194 ceph-mon[36445]: pgmap v234: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-03-16T10:16:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:51 smithi039 ceph-mon[31510]: pgmap v234: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.2 MiB/s wr, 472 op/s 2024-03-16T10:16:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 430.0 s, 496.4 tps, lat 201.733 ms stddev 235.793 2024-03-16T10:16:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:53 smithi105 ceph-mon[36559]: pgmap v235: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-03-16T10:16:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:53 smithi194 ceph-mon[36445]: pgmap v235: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-03-16T10:16:53.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:53 smithi039 ceph-mon[31510]: pgmap v235: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.8 MiB/s wr, 558 op/s 2024-03-16T10:16:55.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:55 smithi194 ceph-mon[36445]: pgmap v236: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-03-16T10:16:55.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:55 smithi039 ceph-mon[31510]: pgmap v236: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-03-16T10:16:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:55 smithi105 ceph-mon[36559]: pgmap v236: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.0 MiB/s wr, 454 op/s 2024-03-16T10:16:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:57 smithi194 ceph-mon[36445]: pgmap v237: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-03-16T10:16:57.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:57 smithi039 ceph-mon[31510]: pgmap v237: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-03-16T10:16:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:57 smithi105 ceph-mon[36559]: pgmap v237: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-03-16T10:16:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 435.0 s, 493.2 tps, lat 201.957 ms stddev 245.846 2024-03-16T10:16:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:16:59 smithi194 ceph-mon[36445]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.5 MiB/s wr, 527 op/s 2024-03-16T10:16:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:16:59 smithi039 ceph-mon[31510]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.5 MiB/s wr, 527 op/s 2024-03-16T10:16:59.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:16:59 smithi105 ceph-mon[36559]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.5 MiB/s wr, 527 op/s 2024-03-16T10:17:01.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:01 smithi039 ceph-mon[31510]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-03-16T10:17:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:01 smithi194 ceph-mon[36445]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-03-16T10:17:01.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:01 smithi105 ceph-mon[36559]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.5 MiB/s wr, 452 op/s 2024-03-16T10:17:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 440.0 s, 465.8 tps, lat 206.088 ms stddev 364.368 2024-03-16T10:17:03.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:03 smithi194 ceph-mon[36445]: pgmap v240: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-03-16T10:17:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:03 smithi039 ceph-mon[31510]: pgmap v240: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-03-16T10:17:03.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:03 smithi105 ceph-mon[36559]: pgmap v240: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 6.0 MiB/s wr, 553 op/s 2024-03-16T10:17:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:05 smithi194 ceph-mon[36445]: pgmap v241: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-16T10:17:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:05 smithi194 ceph-mon[36445]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T10:17:05.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:05 smithi039 ceph-mon[31510]: pgmap v241: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-16T10:17:05.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:05 smithi039 ceph-mon[31510]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T10:17:05.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:05 smithi105 ceph-mon[36559]: pgmap v241: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-16T10:17:05.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:05 smithi105 ceph-mon[36559]: osdmap e99: 12 total, 12 up, 12 in 2024-03-16T10:17:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:07 smithi194 ceph-mon[36445]: pgmap v243: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-03-16T10:17:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:07 smithi039 ceph-mon[31510]: pgmap v243: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-03-16T10:17:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:07 smithi105 ceph-mon[36559]: pgmap v243: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-03-16T10:17:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 445.0 s, 451.2 tps, lat 229.062 ms stddev 295.912 2024-03-16T10:17:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:09 smithi194 ceph-mon[36445]: pgmap v244: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:17:09.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:09 smithi039 ceph-mon[31510]: pgmap v244: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:17:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:09 smithi105 ceph-mon[36559]: pgmap v244: 129 pgs: 2 active+clean+snaptrim, 123 active+clean, 4 active+clean+snaptrim_wait; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-03-16T10:17:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:11 smithi194 ceph-mon[36445]: pgmap v245: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.7 MiB/s wr, 458 op/s 2024-03-16T10:17:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:11 smithi039 ceph-mon[31510]: pgmap v245: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.7 MiB/s wr, 458 op/s 2024-03-16T10:17:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:11 smithi105 ceph-mon[36559]: pgmap v245: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 4.7 MiB/s wr, 458 op/s 2024-03-16T10:17:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:13 smithi194 ceph-mon[36445]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T10:17:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:13 smithi194 ceph-mon[36445]: pgmap v247: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-03-16T10:17:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 450.0 s, 528.3 tps, lat 190.454 ms stddev 237.022 2024-03-16T10:17:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:13 smithi039 ceph-mon[31510]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T10:17:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:13 smithi039 ceph-mon[31510]: pgmap v247: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-03-16T10:17:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:13 smithi105 ceph-mon[36559]: osdmap e100: 12 total, 12 up, 12 in 2024-03-16T10:17:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:13 smithi105 ceph-mon[36559]: pgmap v247: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2024-03-16T10:17:15.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:15 smithi039 ceph-mon[31510]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-03-16T10:17:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:15 smithi105 ceph-mon[36559]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-03-16T10:17:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:15 smithi194 ceph-mon[36445]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.8 MiB/s wr, 478 op/s 2024-03-16T10:17:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:17 smithi039 ceph-mon[31510]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-03-16T10:17:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:17 smithi105 ceph-mon[36559]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-03-16T10:17:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:17 smithi194 ceph-mon[36445]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.4 MiB/s wr, 447 op/s 2024-03-16T10:17:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 455.0 s, 487.5 tps, lat 207.317 ms stddev 230.454 2024-03-16T10:17:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:19 smithi039 ceph-mon[31510]: pgmap v250: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-03-16T10:17:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:19 smithi105 ceph-mon[36559]: pgmap v250: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-03-16T10:17:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:19 smithi194 ceph-mon[36445]: pgmap v250: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-03-16T10:17:21.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:21 smithi039 ceph-mon[31510]: pgmap v251: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-03-16T10:17:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:21 smithi105 ceph-mon[36559]: pgmap v251: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-03-16T10:17:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:21 smithi194 ceph-mon[36445]: pgmap v251: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.6 MiB/s wr, 464 op/s 2024-03-16T10:17:23.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:23 smithi194 ceph-mon[36445]: pgmap v252: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-03-16T10:17:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 460.0 s, 495.6 tps, lat 201.162 ms stddev 249.583 2024-03-16T10:17:23.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:23 smithi039 ceph-mon[31510]: pgmap v252: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-03-16T10:17:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:23 smithi105 ceph-mon[36559]: pgmap v252: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 MiB/s wr, 517 op/s 2024-03-16T10:17:25.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:25 smithi194 ceph-mon[36445]: pgmap v253: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-03-16T10:17:25.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:25 smithi039 ceph-mon[31510]: pgmap v253: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-03-16T10:17:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:25 smithi105 ceph-mon[36559]: pgmap v253: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.6 MiB/s wr, 459 op/s 2024-03-16T10:17:27.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:27 smithi039 ceph-mon[31510]: pgmap v254: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-03-16T10:17:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:27 smithi105 ceph-mon[36559]: pgmap v254: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-03-16T10:17:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:27 smithi194 ceph-mon[36445]: pgmap v254: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 454 op/s 2024-03-16T10:17:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 465.0 s, 487.8 tps, lat 206.653 ms stddev 225.090 2024-03-16T10:17:29.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:29 smithi039 ceph-mon[31510]: pgmap v255: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-03-16T10:17:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:29 smithi105 ceph-mon[36559]: pgmap v255: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-03-16T10:17:29.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:29 smithi194 ceph-mon[36445]: pgmap v255: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-03-16T10:17:31.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:31 smithi039 ceph-mon[31510]: pgmap v256: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-03-16T10:17:31.506 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:31 smithi194 ceph-mon[36445]: pgmap v256: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-03-16T10:17:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:31 smithi105 ceph-mon[36559]: pgmap v256: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-03-16T10:17:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 470.0 s, 538.2 tps, lat 182.361 ms stddev 208.212 2024-03-16T10:17:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:33 smithi105 ceph-mon[36559]: pgmap v257: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-03-16T10:17:33.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:33 smithi194 ceph-mon[36445]: pgmap v257: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-03-16T10:17:33.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:33 smithi039 ceph-mon[31510]: pgmap v257: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 564 op/s 2024-03-16T10:17:35.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:35 smithi105 ceph-mon[36559]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-03-16T10:17:35.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:35 smithi194 ceph-mon[36445]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-03-16T10:17:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:35 smithi039 ceph-mon[31510]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 469 op/s 2024-03-16T10:17:37.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:37 smithi105 ceph-mon[36559]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-03-16T10:17:37.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:37 smithi194 ceph-mon[36445]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-03-16T10:17:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:37 smithi039 ceph-mon[31510]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 485 op/s 2024-03-16T10:17:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 475.0 s, 456.2 tps, lat 214.332 ms stddev 241.533 2024-03-16T10:17:38.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:38 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:17:38.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:38 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:17:38.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:38 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:17:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:39 smithi105 ceph-mon[36559]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-03-16T10:17:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:39 smithi194 ceph-mon[36445]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-03-16T10:17:39.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:39 smithi039 ceph-mon[31510]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.1 MiB/s wr, 553 op/s 2024-03-16T10:17:40.627 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:40 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:17:40.627 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:40 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:17:40.627 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:40 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:17:40.760 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:40 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:17:40.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:40 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:17:40.761 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:40 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:17:40.786 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:40 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:17:40.786 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:40 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:17:40.787 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:40 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:17:41.621 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:41 smithi105 ceph-mon[36559]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:17:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:41 smithi194 ceph-mon[36445]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:17:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:41 smithi039 ceph-mon[31510]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:17:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 480.0 s, 440.2 tps, lat 233.491 ms stddev 290.235 2024-03-16T10:17:43.556 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:43 smithi039 ceph-mon[31510]: pgmap v262: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-03-16T10:17:43.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:43 smithi105 ceph-mon[36559]: pgmap v262: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-03-16T10:17:43.908 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:43 smithi194 ceph-mon[36445]: pgmap v262: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 557 op/s 2024-03-16T10:17:45.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:45 smithi105 ceph-mon[36559]: pgmap v263: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:17:45.710 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:45 smithi194 ceph-mon[36445]: pgmap v263: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:17:45.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:45 smithi039 ceph-mon[31510]: pgmap v263: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:17:47.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:47 smithi105 ceph-mon[36559]: pgmap v264: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-03-16T10:17:47.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:47 smithi194 ceph-mon[36445]: pgmap v264: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-03-16T10:17:47.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:47 smithi039 ceph-mon[31510]: pgmap v264: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 457 op/s 2024-03-16T10:17:48.489 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 485.0 s, 425.6 tps, lat 228.721 ms stddev 264.934 2024-03-16T10:17:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:49 smithi194 ceph-mon[36445]: pgmap v265: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-03-16T10:17:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:49 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:49.730 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:49 smithi105 ceph-mon[36559]: pgmap v265: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-03-16T10:17:49.730 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:49 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:49 smithi039 ceph-mon[31510]: pgmap v265: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-03-16T10:17:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:49 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:51.630 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:51 smithi105 ceph-mon[36559]: pgmap v266: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-03-16T10:17:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:51 smithi194 ceph-mon[36445]: pgmap v266: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-03-16T10:17:51.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:51 smithi039 ceph-mon[31510]: pgmap v266: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.8 MiB/s wr, 403 op/s 2024-03-16T10:17:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 490.0 s, 447.0 tps, lat 224.215 ms stddev 250.476 2024-03-16T10:17:53.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:53 smithi105 ceph-mon[36559]: pgmap v267: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-03-16T10:17:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:53 smithi194 ceph-mon[36445]: pgmap v267: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-03-16T10:17:53.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:53 smithi039 ceph-mon[31510]: pgmap v267: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-03-16T10:17:54.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:54 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:54.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:54 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:54.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:54 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:17:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:55 smithi194 ceph-mon[36445]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-16T10:17:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:55 smithi039 ceph-mon[31510]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-16T10:17:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:55 smithi105 ceph-mon[36559]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 3.9 MiB/s wr, 431 op/s 2024-03-16T10:17:57.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:57 smithi194 ceph-mon[36445]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-03-16T10:17:57.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:57 smithi039 ceph-mon[31510]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-03-16T10:17:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:57 smithi105 ceph-mon[36559]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 4.1 MiB/s wr, 426 op/s 2024-03-16T10:17:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 495.0 s, 426.6 tps, lat 236.250 ms stddev 368.323 2024-03-16T10:17:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:17:59 smithi194 ceph-mon[36445]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 4.5 MiB/s wr, 476 op/s 2024-03-16T10:17:59.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:17:59 smithi039 ceph-mon[31510]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 4.5 MiB/s wr, 476 op/s 2024-03-16T10:17:59.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:17:59 smithi105 ceph-mon[36559]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 4.5 MiB/s wr, 476 op/s 2024-03-16T10:18:01.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:01 smithi194 ceph-mon[36445]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-03-16T10:18:01.936 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:01 smithi105 ceph-mon[36559]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-03-16T10:18:01.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:01 smithi039 ceph-mon[31510]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 3.9 MiB/s wr, 401 op/s 2024-03-16T10:18:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 500.0 s, 394.6 tps, lat 254.788 ms stddev 296.831 2024-03-16T10:18:03.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:03 smithi194 ceph-mon[36445]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-03-16T10:18:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:03 smithi039 ceph-mon[31510]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-03-16T10:18:03.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:03 smithi105 ceph-mon[36559]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-03-16T10:18:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:05 smithi194 ceph-mon[36445]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-03-16T10:18:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:05 smithi194 ceph-mon[36445]: osdmap e101: 12 total, 12 up, 12 in 2024-03-16T10:18:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:05 smithi039 ceph-mon[31510]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-03-16T10:18:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:05 smithi039 ceph-mon[31510]: osdmap e101: 12 total, 12 up, 12 in 2024-03-16T10:18:05.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:05 smithi105 ceph-mon[36559]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 3.8 MiB/s wr, 411 op/s 2024-03-16T10:18:05.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:05 smithi105 ceph-mon[36559]: osdmap e101: 12 total, 12 up, 12 in 2024-03-16T10:18:07.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:07 smithi194 ceph-mon[36445]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-03-16T10:18:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:07 smithi039 ceph-mon[31510]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-03-16T10:18:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:07 smithi105 ceph-mon[36559]: pgmap v275: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 3.9 MiB/s wr, 421 op/s 2024-03-16T10:18:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 505.0 s, 441.6 tps, lat 225.937 ms stddev 315.910 2024-03-16T10:18:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:09 smithi194 ceph-mon[36445]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-03-16T10:18:09.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:09 smithi039 ceph-mon[31510]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-03-16T10:18:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:09 smithi105 ceph-mon[36559]: pgmap v276: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-03-16T10:18:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:11 smithi194 ceph-mon[36445]: pgmap v277: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-03-16T10:18:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:11 smithi039 ceph-mon[31510]: pgmap v277: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-03-16T10:18:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:11 smithi105 ceph-mon[36559]: pgmap v277: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 3.9 MiB/s wr, 417 op/s 2024-03-16T10:18:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:13 smithi194 ceph-mon[36445]: osdmap e102: 12 total, 12 up, 12 in 2024-03-16T10:18:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:13 smithi194 ceph-mon[36445]: pgmap v279: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-03-16T10:18:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 510.0 s, 460.1 tps, lat 215.045 ms stddev 257.492 2024-03-16T10:18:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:13 smithi039 ceph-mon[31510]: osdmap e102: 12 total, 12 up, 12 in 2024-03-16T10:18:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:13 smithi039 ceph-mon[31510]: pgmap v279: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-03-16T10:18:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:13 smithi105 ceph-mon[36559]: osdmap e102: 12 total, 12 up, 12 in 2024-03-16T10:18:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:13 smithi105 ceph-mon[36559]: pgmap v279: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-03-16T10:18:15.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:15 smithi039 ceph-mon[31510]: pgmap v280: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 460 op/s 2024-03-16T10:18:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:15 smithi105 ceph-mon[36559]: pgmap v280: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 460 op/s 2024-03-16T10:18:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:15 smithi194 ceph-mon[36445]: pgmap v280: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.3 MiB/s wr, 460 op/s 2024-03-16T10:18:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:17 smithi039 ceph-mon[31510]: pgmap v281: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-03-16T10:18:17.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:17 smithi039 ceph-mon[31510]: osdmap e103: 12 total, 12 up, 12 in 2024-03-16T10:18:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:17 smithi105 ceph-mon[36559]: pgmap v281: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-03-16T10:18:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:17 smithi105 ceph-mon[36559]: osdmap e103: 12 total, 12 up, 12 in 2024-03-16T10:18:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:17 smithi194 ceph-mon[36445]: pgmap v281: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 425 op/s 2024-03-16T10:18:17.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:17 smithi194 ceph-mon[36445]: osdmap e103: 12 total, 12 up, 12 in 2024-03-16T10:18:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 515.0 s, 415.5 tps, lat 233.367 ms stddev 292.208 2024-03-16T10:18:19.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:19 smithi039 ceph-mon[31510]: pgmap v283: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-16T10:18:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:19 smithi105 ceph-mon[36559]: pgmap v283: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-16T10:18:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:19 smithi194 ceph-mon[36445]: pgmap v283: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-16T10:18:21.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:21 smithi039 ceph-mon[31510]: pgmap v284: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-03-16T10:18:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:21 smithi105 ceph-mon[36559]: pgmap v284: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-03-16T10:18:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:21 smithi194 ceph-mon[36445]: pgmap v284: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.6 MiB/s wr, 380 op/s 2024-03-16T10:18:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 520.0 s, 489.2 tps, lat 213.416 ms stddev 264.164 2024-03-16T10:18:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:23 smithi105 ceph-mon[36559]: pgmap v285: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:18:23.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:23 smithi194 ceph-mon[36445]: pgmap v285: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:18:23.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:23 smithi039 ceph-mon[31510]: pgmap v285: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-03-16T10:18:25.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:25 smithi105 ceph-mon[36559]: pgmap v286: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-16T10:18:25.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:25 smithi194 ceph-mon[36445]: pgmap v286: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-16T10:18:25.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:25 smithi039 ceph-mon[31510]: pgmap v286: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-03-16T10:18:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:27 smithi105 ceph-mon[36559]: pgmap v287: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-16T10:18:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:27 smithi194 ceph-mon[36445]: pgmap v287: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-16T10:18:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:27 smithi039 ceph-mon[31510]: pgmap v287: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.1 MiB/s wr, 452 op/s 2024-03-16T10:18:28.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 525.0 s, 436.0 tps, lat 228.199 ms stddev 293.394 2024-03-16T10:18:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:29 smithi105 ceph-mon[36559]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-03-16T10:18:29.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:29 smithi194 ceph-mon[36445]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-03-16T10:18:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:29 smithi039 ceph-mon[31510]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 4.0 MiB/s wr, 436 op/s 2024-03-16T10:18:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:31 smithi105 ceph-mon[36559]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:31.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:31 smithi194 ceph-mon[36445]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:31 smithi039 ceph-mon[31510]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 530.0 s, 460.4 tps, lat 219.771 ms stddev 262.491 2024-03-16T10:18:33.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:33 smithi105 ceph-mon[36559]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-03-16T10:18:33.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:33 smithi194 ceph-mon[36445]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-03-16T10:18:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:33 smithi039 ceph-mon[31510]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.8 MiB/s wr, 518 op/s 2024-03-16T10:18:35.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:35 smithi105 ceph-mon[36559]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-03-16T10:18:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:35 smithi194 ceph-mon[36445]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-03-16T10:18:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:35 smithi039 ceph-mon[31510]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.9 MiB/s wr, 427 op/s 2024-03-16T10:18:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:37 smithi105 ceph-mon[36559]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:37.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:37 smithi194 ceph-mon[36445]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:37 smithi039 ceph-mon[31510]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-03-16T10:18:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 535.0 s, 446.0 tps, lat 225.900 ms stddev 258.625 2024-03-16T10:18:39.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:39 smithi105 ceph-mon[36559]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-03-16T10:18:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:39 smithi194 ceph-mon[36445]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-03-16T10:18:39.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:39 smithi039 ceph-mon[31510]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.5 MiB/s wr, 499 op/s 2024-03-16T10:18:41.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:41 smithi105 ceph-mon[36559]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 422 op/s 2024-03-16T10:18:41.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:41 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:18:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:41 smithi194 ceph-mon[36445]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 422 op/s 2024-03-16T10:18:41.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:41 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:18:41.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:41 smithi039 ceph-mon[31510]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.0 MiB/s wr, 422 op/s 2024-03-16T10:18:41.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:41 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:18:42.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:42 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:18:42.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:42 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:18:42.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:42 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:18:42.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:42 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:18:42.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:42 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:18:42.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:42 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:18:42.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:42 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:18:42.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:42 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:18:42.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:42 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:18:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 540.0 s, 413.8 tps, lat 237.164 ms stddev 290.976 2024-03-16T10:18:43.658 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:43 smithi039 ceph-mon[31510]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-03-16T10:18:43.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:43 smithi194 ceph-mon[36445]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-03-16T10:18:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:43 smithi105 ceph-mon[36559]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 506 op/s 2024-03-16T10:18:45.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:45 smithi194 ceph-mon[36445]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:45.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:45 smithi039 ceph-mon[31510]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:45 smithi105 ceph-mon[36559]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:47.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:47 smithi194 ceph-mon[36445]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:47 smithi039 ceph-mon[31510]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:47 smithi105 ceph-mon[36559]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-03-16T10:18:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 545.0 s, 430.8 tps, lat 224.172 ms stddev 372.693 2024-03-16T10:18:49.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:49 smithi194 ceph-mon[36445]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-03-16T10:18:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:49 smithi039 ceph-mon[31510]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-03-16T10:18:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:49 smithi105 ceph-mon[36559]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-03-16T10:18:51.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:51 smithi194 ceph-mon[36445]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-16T10:18:51.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:51 smithi039 ceph-mon[31510]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-16T10:18:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:51 smithi105 ceph-mon[36559]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.9 MiB/s wr, 414 op/s 2024-03-16T10:18:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 550.0 s, 431.2 tps, lat 229.842 ms stddev 332.970 2024-03-16T10:18:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:53 smithi194 ceph-mon[36445]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-03-16T10:18:53.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:53 smithi039 ceph-mon[31510]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-03-16T10:18:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:53 smithi105 ceph-mon[36559]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.1 MiB/s wr, 478 op/s 2024-03-16T10:18:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:55 smithi194 ceph-mon[36445]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-03-16T10:18:55.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:55 smithi039 ceph-mon[31510]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-03-16T10:18:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:55 smithi105 ceph-mon[36559]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.3 MiB/s wr, 391 op/s 2024-03-16T10:18:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:57 smithi194 ceph-mon[36445]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-03-16T10:18:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:57 smithi039 ceph-mon[31510]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-03-16T10:18:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:57 smithi105 ceph-mon[36559]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-03-16T10:18:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 555.0 s, 506.8 tps, lat 206.338 ms stddev 260.468 2024-03-16T10:18:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:18:59 smithi194 ceph-mon[36445]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-03-16T10:18:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:18:59 smithi039 ceph-mon[31510]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-03-16T10:18:59.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:18:59 smithi105 ceph-mon[36559]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-03-16T10:19:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:01 smithi194 ceph-mon[36445]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-03-16T10:19:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:01 smithi039 ceph-mon[31510]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-03-16T10:19:01.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:01 smithi105 ceph-mon[36559]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.8 MiB/s wr, 419 op/s 2024-03-16T10:19:03.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 560.0 s, 381.2 tps, lat 261.466 ms stddev 346.965 2024-03-16T10:19:03.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:03 smithi039 ceph-mon[31510]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-03-16T10:19:03.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:03 smithi105 ceph-mon[36559]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-03-16T10:19:03.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:03 smithi194 ceph-mon[36445]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.2 MiB/s wr, 499 op/s 2024-03-16T10:19:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:05 smithi039 ceph-mon[31510]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-03-16T10:19:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:05 smithi039 ceph-mon[31510]: osdmap e104: 12 total, 12 up, 12 in 2024-03-16T10:19:05.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:05 smithi105 ceph-mon[36559]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-03-16T10:19:05.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:05 smithi105 ceph-mon[36559]: osdmap e104: 12 total, 12 up, 12 in 2024-03-16T10:19:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:05 smithi194 ceph-mon[36445]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-03-16T10:19:05.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:05 smithi194 ceph-mon[36445]: osdmap e104: 12 total, 12 up, 12 in 2024-03-16T10:19:07.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:07 smithi039 ceph-mon[31510]: pgmap v308: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:07 smithi105 ceph-mon[36559]: pgmap v308: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:07.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:07 smithi194 ceph-mon[36445]: pgmap v308: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 565.0 s, 414.0 tps, lat 247.958 ms stddev 347.862 2024-03-16T10:19:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:09 smithi039 ceph-mon[31510]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-03-16T10:19:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:09 smithi105 ceph-mon[36559]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-03-16T10:19:09.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:09 smithi194 ceph-mon[36445]: pgmap v309: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.9 MiB/s wr, 437 op/s 2024-03-16T10:19:11.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:11 smithi039 ceph-mon[31510]: pgmap v310: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-03-16T10:19:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:11 smithi105 ceph-mon[36559]: pgmap v310: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-03-16T10:19:11.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:11 smithi194 ceph-mon[36445]: pgmap v310: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-03-16T10:19:13.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:13 smithi194 ceph-mon[36445]: osdmap e105: 12 total, 12 up, 12 in 2024-03-16T10:19:13.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:13 smithi194 ceph-mon[36445]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.0 MiB/s wr, 444 op/s 2024-03-16T10:19:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 570.0 s, 479.4 tps, lat 200.913 ms stddev 271.518 2024-03-16T10:19:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:13 smithi039 ceph-mon[31510]: osdmap e105: 12 total, 12 up, 12 in 2024-03-16T10:19:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:13 smithi039 ceph-mon[31510]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.0 MiB/s wr, 444 op/s 2024-03-16T10:19:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:13 smithi105 ceph-mon[36559]: osdmap e105: 12 total, 12 up, 12 in 2024-03-16T10:19:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:13 smithi105 ceph-mon[36559]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.0 MiB/s wr, 444 op/s 2024-03-16T10:19:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:15 smithi105 ceph-mon[36559]: pgmap v313: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-03-16T10:19:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:15 smithi194 ceph-mon[36445]: pgmap v313: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-03-16T10:19:15.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:15 smithi039 ceph-mon[31510]: pgmap v313: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-03-16T10:19:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:17 smithi105 ceph-mon[36559]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-03-16T10:19:17.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:17 smithi194 ceph-mon[36445]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-03-16T10:19:17.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:17 smithi039 ceph-mon[31510]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-03-16T10:19:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 575.0 s, 453.8 tps, lat 220.928 ms stddev 263.346 2024-03-16T10:19:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:19 smithi105 ceph-mon[36559]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:19.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:19 smithi194 ceph-mon[36445]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:19 smithi039 ceph-mon[31510]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-03-16T10:19:21.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:21 smithi105 ceph-mon[36559]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-16T10:19:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:21 smithi194 ceph-mon[36445]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-16T10:19:21.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:21 smithi039 ceph-mon[31510]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 471 op/s 2024-03-16T10:19:22.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:22 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:22.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:22 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:22.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:22 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 580.0 s, 431.0 tps, lat 224.586 ms stddev 262.493 2024-03-16T10:19:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:23 smithi105 ceph-mon[36559]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-03-16T10:19:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:23 smithi194 ceph-mon[36445]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-03-16T10:19:23.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:23 smithi039 ceph-mon[31510]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-03-16T10:19:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:25 smithi105 ceph-mon[36559]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-03-16T10:19:25.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:25 smithi194 ceph-mon[36445]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-03-16T10:19:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:25 smithi039 ceph-mon[31510]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.9 MiB/s wr, 444 op/s 2024-03-16T10:19:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:27 smithi105 ceph-mon[36559]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-03-16T10:19:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:27 smithi194 ceph-mon[36445]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-03-16T10:19:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:27 smithi039 ceph-mon[31510]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 3.6 MiB/s wr, 424 op/s 2024-03-16T10:19:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 585.0 s, 482.2 tps, lat 216.189 ms stddev 236.527 2024-03-16T10:19:29.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:29 smithi105 ceph-mon[36559]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-16T10:19:29.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:29 smithi194 ceph-mon[36445]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-16T10:19:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:29 smithi039 ceph-mon[31510]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-03-16T10:19:31.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:31 smithi105 ceph-mon[36559]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-03-16T10:19:31.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:31 smithi194 ceph-mon[36445]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-03-16T10:19:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:31 smithi039 ceph-mon[31510]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.8 MiB/s wr, 429 op/s 2024-03-16T10:19:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 590.0 s, 464.0 tps, lat 216.410 ms stddev 249.753 2024-03-16T10:19:33.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:33 smithi194 ceph-mon[36445]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-03-16T10:19:33.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:33 smithi039 ceph-mon[31510]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-03-16T10:19:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:33 smithi105 ceph-mon[36559]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 503 op/s 2024-03-16T10:19:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:35 smithi194 ceph-mon[36445]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-03-16T10:19:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:35 smithi039 ceph-mon[31510]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-03-16T10:19:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:35 smithi105 ceph-mon[36559]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-03-16T10:19:37.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:37 smithi194 ceph-mon[36445]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-16T10:19:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:37 smithi039 ceph-mon[31510]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-16T10:19:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:37 smithi105 ceph-mon[36559]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2024-03-16T10:19:38.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 595.0 s, 425.4 tps, lat 242.271 ms stddev 300.685 2024-03-16T10:19:39.666 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:39 smithi105 ceph-mon[36559]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.9 MiB/s wr, 542 op/s 2024-03-16T10:19:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:39 smithi194 ceph-mon[36445]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.9 MiB/s wr, 542 op/s 2024-03-16T10:19:39.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:39 smithi039 ceph-mon[31510]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.9 MiB/s wr, 542 op/s 2024-03-16T10:19:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:41 smithi194 ceph-mon[36445]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.8 MiB/s wr, 466 op/s 2024-03-16T10:19:41.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:41 smithi039 ceph-mon[31510]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.8 MiB/s wr, 466 op/s 2024-03-16T10:19:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:41 smithi105 ceph-mon[36559]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.8 MiB/s wr, 466 op/s 2024-03-16T10:19:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 600.0 s, 415.6 tps, lat 231.094 ms stddev 238.502 2024-03-16T10:19:43.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:43 smithi194 ceph-mon[36445]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.8 MiB/s wr, 523 op/s 2024-03-16T10:19:43.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:43 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:19:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:43 smithi039 ceph-mon[31510]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.8 MiB/s wr, 523 op/s 2024-03-16T10:19:43.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:43 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:19:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:43 smithi105 ceph-mon[36559]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.8 MiB/s wr, 523 op/s 2024-03-16T10:19:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:43 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:44 smithi105 ceph-mon[36559]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 437 op/s 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:44.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:44 smithi194 ceph-mon[36445]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 437 op/s 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:45.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:44 smithi039 ceph-mon[31510]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.1 MiB/s wr, 437 op/s 2024-03-16T10:19:47.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:47 smithi039 ceph-mon[31510]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 460 op/s 2024-03-16T10:19:47.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:47 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:47 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:47 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:47 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:47 smithi194 ceph-mon[36445]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 460 op/s 2024-03-16T10:19:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:47 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:47 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:47 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:47 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:47 smithi105 ceph-mon[36559]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 460 op/s 2024-03-16T10:19:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:47 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:47 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:47 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:47 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:48.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 605.0 s, 396.4 tps, lat 252.042 ms stddev 287.960 2024-03-16T10:19:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:19:48.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:48 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:19:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:19:48.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:48 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:19:48.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi194 to 174.1M 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi194 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-16T10:19:48.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:48.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-16T10:19:48.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:48 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi105 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-16T10:19:49.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:49 smithi194 ceph-mon[36445]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.2 MiB/s wr, 491 op/s 2024-03-16T10:19:49.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:49 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:49 smithi039 ceph-mon[31510]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.2 MiB/s wr, 491 op/s 2024-03-16T10:19:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:49 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:49.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:49 smithi105 ceph-mon[36559]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.2 MiB/s wr, 491 op/s 2024-03-16T10:19:49.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:49 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:19:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:50 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:50 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:50 smithi105 ceph-mon[36559]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 393 op/s 2024-03-16T10:19:51.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:50 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:50 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:50 smithi194 ceph-mon[36445]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 393 op/s 2024-03-16T10:19:51.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:50 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:50 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:51.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:50 smithi039 ceph-mon[31510]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 393 op/s 2024-03-16T10:19:52.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:19:52.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:51 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:19:52.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:19:52.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:51 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:19:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:19:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:19:52.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:51 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:19:53.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:52 smithi194 ceph-mon[36445]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:19:53.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:52 smithi194 ceph-mon[36445]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:19:53.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:52 smithi194 ceph-mon[36445]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.2 MiB/s wr, 426 op/s 2024-03-16T10:19:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:52 smithi039 ceph-mon[31510]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:19:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:52 smithi039 ceph-mon[31510]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:19:53.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:52 smithi039 ceph-mon[31510]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.2 MiB/s wr, 426 op/s 2024-03-16T10:19:53.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:52 smithi105 ceph-mon[36559]: Adjusting osd_memory_target on smithi039 to 176.8M 2024-03-16T10:19:53.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:52 smithi105 ceph-mon[36559]: Unable to set osd_memory_target on smithi039 to 185393152: error parsing value: Value '185393152' is below minimum 939524096 2024-03-16T10:19:53.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:52 smithi105 ceph-mon[36559]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.2 MiB/s wr, 426 op/s 2024-03-16T10:19:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 610.0 s, 453.4 tps, lat 225.879 ms stddev 245.473 2024-03-16T10:19:55.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:55 smithi105 ceph-mon[36559]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-03-16T10:19:55.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:55 smithi194 ceph-mon[36445]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-03-16T10:19:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:55 smithi039 ceph-mon[31510]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-03-16T10:19:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:57 smithi105 ceph-mon[36559]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-03-16T10:19:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:57 smithi194 ceph-mon[36445]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-03-16T10:19:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:57 smithi039 ceph-mon[31510]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.0 MiB/s wr, 385 op/s 2024-03-16T10:19:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 615.0 s, 458.2 tps, lat 210.894 ms stddev 270.967 2024-03-16T10:19:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:19:59 smithi105 ceph-mon[36559]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-03-16T10:19:59.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:19:59 smithi194 ceph-mon[36445]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-03-16T10:19:59.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:19:59 smithi039 ceph-mon[31510]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-03-16T10:20:00.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:00 smithi105 ceph-mon[36559]: overall HEALTH_OK 2024-03-16T10:20:00.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:00 smithi194 ceph-mon[36445]: overall HEALTH_OK 2024-03-16T10:20:00.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:00 smithi039 ceph-mon[31510]: overall HEALTH_OK 2024-03-16T10:20:01.606 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:01 smithi194 ceph-mon[36445]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-03-16T10:20:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:01 smithi105 ceph-mon[36559]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-03-16T10:20:01.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:01 smithi039 ceph-mon[31510]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.1 MiB/s wr, 381 op/s 2024-03-16T10:20:03.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 620.0 s, 413.0 tps, lat 239.996 ms stddev 325.211 2024-03-16T10:20:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:03 smithi105 ceph-mon[36559]: pgmap v337: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 436 op/s 2024-03-16T10:20:03.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:03 smithi194 ceph-mon[36445]: pgmap v337: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 436 op/s 2024-03-16T10:20:03.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:03 smithi039 ceph-mon[31510]: pgmap v337: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 436 op/s 2024-03-16T10:20:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:05 smithi105 ceph-mon[36559]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-03-16T10:20:05.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:05 smithi194 ceph-mon[36445]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-03-16T10:20:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:05 smithi039 ceph-mon[31510]: pgmap v338: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 404 op/s 2024-03-16T10:20:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:07 smithi105 ceph-mon[36559]: pgmap v339: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 427 op/s 2024-03-16T10:20:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:07 smithi194 ceph-mon[36445]: pgmap v339: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 427 op/s 2024-03-16T10:20:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:07 smithi039 ceph-mon[31510]: pgmap v339: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 427 op/s 2024-03-16T10:20:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 625.0 s, 502.0 tps, lat 202.907 ms stddev 234.867 2024-03-16T10:20:09.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:09 smithi105 ceph-mon[36559]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-03-16T10:20:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:09 smithi194 ceph-mon[36445]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-03-16T10:20:09.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:09 smithi039 ceph-mon[31510]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-03-16T10:20:10.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:10 smithi194 ceph-mon[36445]: osdmap e106: 12 total, 12 up, 12 in 2024-03-16T10:20:10.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:10 smithi039 ceph-mon[31510]: osdmap e106: 12 total, 12 up, 12 in 2024-03-16T10:20:10.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:10 smithi105 ceph-mon[36559]: osdmap e106: 12 total, 12 up, 12 in 2024-03-16T10:20:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:11 smithi194 ceph-mon[36445]: pgmap v342: 129 pgs: 3 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.1 MiB/s wr, 429 op/s 2024-03-16T10:20:11.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:11 smithi039 ceph-mon[31510]: pgmap v342: 129 pgs: 3 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.1 MiB/s wr, 429 op/s 2024-03-16T10:20:11.757 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:11 smithi105 ceph-mon[36559]: pgmap v342: 129 pgs: 3 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.1 MiB/s wr, 429 op/s 2024-03-16T10:20:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 630.0 s, 373.4 tps, lat 273.101 ms stddev 368.269 2024-03-16T10:20:13.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:13 smithi105 ceph-mon[36559]: pgmap v343: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-03-16T10:20:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:13 smithi194 ceph-mon[36445]: pgmap v343: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-03-16T10:20:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:13 smithi039 ceph-mon[31510]: pgmap v343: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.3 MiB/s wr, 406 op/s 2024-03-16T10:20:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:15 smithi194 ceph-mon[36445]: pgmap v344: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-03-16T10:20:15.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:15 smithi039 ceph-mon[31510]: pgmap v344: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-03-16T10:20:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:15 smithi105 ceph-mon[36559]: pgmap v344: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-03-16T10:20:17.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:17 smithi194 ceph-mon[36445]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 433 op/s 2024-03-16T10:20:17.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:17 smithi194 ceph-mon[36445]: osdmap e107: 12 total, 12 up, 12 in 2024-03-16T10:20:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:17 smithi039 ceph-mon[31510]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 433 op/s 2024-03-16T10:20:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:17 smithi039 ceph-mon[31510]: osdmap e107: 12 total, 12 up, 12 in 2024-03-16T10:20:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:17 smithi105 ceph-mon[36559]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.2 MiB/s wr, 433 op/s 2024-03-16T10:20:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:17 smithi105 ceph-mon[36559]: osdmap e107: 12 total, 12 up, 12 in 2024-03-16T10:20:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 635.0 s, 449.2 tps, lat 220.845 ms stddev 258.028 2024-03-16T10:20:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:19 smithi194 ceph-mon[36445]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-16T10:20:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:19 smithi039 ceph-mon[31510]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-16T10:20:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:19 smithi105 ceph-mon[36559]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.3 MiB/s wr, 439 op/s 2024-03-16T10:20:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:21 smithi194 ceph-mon[36445]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.0 MiB/s wr, 396 op/s 2024-03-16T10:20:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:21 smithi039 ceph-mon[31510]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.0 MiB/s wr, 396 op/s 2024-03-16T10:20:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:21 smithi105 ceph-mon[36559]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.0 MiB/s wr, 396 op/s 2024-03-16T10:20:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 640.0 s, 433.8 tps, lat 230.939 ms stddev 296.981 2024-03-16T10:20:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:23 smithi105 ceph-mon[36559]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 429 op/s 2024-03-16T10:20:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:23 smithi194 ceph-mon[36445]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 429 op/s 2024-03-16T10:20:23.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:23 smithi039 ceph-mon[31510]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 429 op/s 2024-03-16T10:20:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:25 smithi194 ceph-mon[36445]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 426 op/s 2024-03-16T10:20:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:25 smithi039 ceph-mon[31510]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 426 op/s 2024-03-16T10:20:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:25 smithi105 ceph-mon[36559]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 426 op/s 2024-03-16T10:20:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:27 smithi194 ceph-mon[36445]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 429 op/s 2024-03-16T10:20:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:27 smithi039 ceph-mon[31510]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 429 op/s 2024-03-16T10:20:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:27 smithi105 ceph-mon[36559]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 429 op/s 2024-03-16T10:20:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 645.0 s, 413.0 tps, lat 237.975 ms stddev 269.702 2024-03-16T10:20:29.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:29 smithi039 ceph-mon[31510]: pgmap v352: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 444 op/s 2024-03-16T10:20:29.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:29 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:20:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:29 smithi105 ceph-mon[36559]: pgmap v352: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 444 op/s 2024-03-16T10:20:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:29 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:20:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:29 smithi194 ceph-mon[36445]: pgmap v352: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.2 MiB/s wr, 444 op/s 2024-03-16T10:20:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:29 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:20:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:31 smithi039 ceph-mon[31510]: pgmap v353: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-16T10:20:31.756 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:31 smithi194 ceph-mon[36445]: pgmap v353: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-16T10:20:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:31 smithi105 ceph-mon[36559]: pgmap v353: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.0 MiB/s wr, 389 op/s 2024-03-16T10:20:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 650.0 s, 388.6 tps, lat 260.844 ms stddev 291.273 2024-03-16T10:20:33.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:33 smithi039 ceph-mon[31510]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-03-16T10:20:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:33 smithi105 ceph-mon[36559]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-03-16T10:20:33.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:33 smithi194 ceph-mon[36445]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-03-16T10:20:35.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:35 smithi039 ceph-mon[31510]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.3 MiB/s wr, 414 op/s 2024-03-16T10:20:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:35 smithi105 ceph-mon[36559]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.3 MiB/s wr, 414 op/s 2024-03-16T10:20:35.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:35 smithi194 ceph-mon[36445]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.3 MiB/s wr, 414 op/s 2024-03-16T10:20:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:37 smithi105 ceph-mon[36559]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-03-16T10:20:37.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:37 smithi194 ceph-mon[36445]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-03-16T10:20:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:37 smithi039 ceph-mon[31510]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-03-16T10:20:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 655.0 s, 405.6 tps, lat 236.963 ms stddev 304.621 2024-03-16T10:20:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:39 smithi105 ceph-mon[36559]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-03-16T10:20:39.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:39 smithi194 ceph-mon[36445]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-03-16T10:20:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:39 smithi039 ceph-mon[31510]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.8 MiB/s wr, 438 op/s 2024-03-16T10:20:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:41 smithi105 ceph-mon[36559]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-03-16T10:20:41.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:41 smithi194 ceph-mon[36445]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-03-16T10:20:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:41 smithi039 ceph-mon[31510]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.8 MiB/s wr, 377 op/s 2024-03-16T10:20:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 660.0 s, 204.6 tps, lat 511.832 ms stddev 974.658 2024-03-16T10:20:43.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:43 smithi039 ceph-mon[31510]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.0 MiB/s wr, 394 op/s 2024-03-16T10:20:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:43 smithi105 ceph-mon[36559]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.0 MiB/s wr, 394 op/s 2024-03-16T10:20:43.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:43 smithi194 ceph-mon[36445]: pgmap v359: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.0 MiB/s wr, 394 op/s 2024-03-16T10:20:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:45 smithi105 ceph-mon[36559]: pgmap v360: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.4 MiB/s wr, 351 op/s 2024-03-16T10:20:45.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:45 smithi194 ceph-mon[36445]: pgmap v360: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.4 MiB/s wr, 351 op/s 2024-03-16T10:20:46.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:45 smithi039 ceph-mon[31510]: pgmap v360: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.4 MiB/s wr, 351 op/s 2024-03-16T10:20:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:47 smithi105 ceph-mon[36559]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-03-16T10:20:47.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:47 smithi194 ceph-mon[36445]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-03-16T10:20:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:47 smithi039 ceph-mon[31510]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.3 MiB/s wr, 345 op/s 2024-03-16T10:20:48.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 665.0 s, 530.8 tps, lat 186.510 ms stddev 211.198 2024-03-16T10:20:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:49 smithi105 ceph-mon[36559]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.8 MiB/s wr, 382 op/s 2024-03-16T10:20:49.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:49 smithi194 ceph-mon[36445]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.8 MiB/s wr, 382 op/s 2024-03-16T10:20:50.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:49 smithi039 ceph-mon[31510]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.8 MiB/s wr, 382 op/s 2024-03-16T10:20:50.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:50 smithi105 ceph-mon[36559]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-03-16T10:20:50.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:50 smithi194 ceph-mon[36445]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-03-16T10:20:51.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:50 smithi039 ceph-mon[31510]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 4.0 MiB/s wr, 319 op/s 2024-03-16T10:20:52.258 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:51 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:20:52.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:51 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:20:52.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:51 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:20:53.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:52 smithi039 ceph-mon[31510]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 367 op/s 2024-03-16T10:20:53.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:52 smithi105 ceph-mon[36559]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 367 op/s 2024-03-16T10:20:53.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:52 smithi194 ceph-mon[36445]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.4 MiB/s wr, 367 op/s 2024-03-16T10:20:53.490 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 670.0 s, 419.7 tps, lat 237.271 ms stddev 323.199 2024-03-16T10:20:54.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:53 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:20:54.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:53 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:20:54.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:53 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:20:54.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:53 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:20:54.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:53 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:20:54.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:53 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:20:54.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:53 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:20:54.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:53 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:20:54.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:53 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:20:55.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:55 smithi039 ceph-mon[31510]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.5 MiB/s wr, 413 op/s 2024-03-16T10:20:55.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:55 smithi105 ceph-mon[36559]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.5 MiB/s wr, 413 op/s 2024-03-16T10:20:55.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:55 smithi194 ceph-mon[36445]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.5 MiB/s wr, 413 op/s 2024-03-16T10:20:57.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:57 smithi105 ceph-mon[36559]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-03-16T10:20:57.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:57 smithi194 ceph-mon[36445]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-03-16T10:20:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:57 smithi039 ceph-mon[31510]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-03-16T10:20:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 675.0 s, 403.1 tps, lat 238.884 ms stddev 269.343 2024-03-16T10:20:59.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:20:59 smithi105 ceph-mon[36559]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 467 op/s 2024-03-16T10:20:59.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:20:59 smithi194 ceph-mon[36445]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 467 op/s 2024-03-16T10:20:59.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:20:59 smithi039 ceph-mon[31510]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.8 MiB/s wr, 467 op/s 2024-03-16T10:21:01.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:01 smithi105 ceph-mon[36559]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-03-16T10:21:01.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:01 smithi194 ceph-mon[36445]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-03-16T10:21:01.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:01 smithi039 ceph-mon[31510]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-03-16T10:21:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 680.0 s, 433.0 tps, lat 240.541 ms stddev 304.385 2024-03-16T10:21:03.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:03 smithi105 ceph-mon[36559]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.0 MiB/s wr, 432 op/s 2024-03-16T10:21:03.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:03 smithi194 ceph-mon[36445]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.0 MiB/s wr, 432 op/s 2024-03-16T10:21:03.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:03 smithi039 ceph-mon[31510]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.0 MiB/s wr, 432 op/s 2024-03-16T10:21:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:05 smithi105 ceph-mon[36559]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 436 op/s 2024-03-16T10:21:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:05 smithi105 ceph-mon[36559]: osdmap e108: 12 total, 12 up, 12 in 2024-03-16T10:21:05.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:05 smithi194 ceph-mon[36445]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 436 op/s 2024-03-16T10:21:05.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:05 smithi194 ceph-mon[36445]: osdmap e108: 12 total, 12 up, 12 in 2024-03-16T10:21:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:05 smithi039 ceph-mon[31510]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 436 op/s 2024-03-16T10:21:05.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:05 smithi039 ceph-mon[31510]: osdmap e108: 12 total, 12 up, 12 in 2024-03-16T10:21:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:07 smithi194 ceph-mon[36445]: pgmap v372: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.8 MiB/s wr, 401 op/s 2024-03-16T10:21:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:07 smithi039 ceph-mon[31510]: pgmap v372: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.8 MiB/s wr, 401 op/s 2024-03-16T10:21:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:07 smithi105 ceph-mon[36559]: pgmap v372: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.8 MiB/s wr, 401 op/s 2024-03-16T10:21:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 685.0 s, 341.0 tps, lat 285.807 ms stddev 432.141 2024-03-16T10:21:09.665 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:09 smithi105 ceph-mon[36559]: pgmap v373: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 MiB/s wr, 404 op/s 2024-03-16T10:21:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:09 smithi194 ceph-mon[36445]: pgmap v373: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 MiB/s wr, 404 op/s 2024-03-16T10:21:09.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:09 smithi039 ceph-mon[31510]: pgmap v373: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.0 MiB/s wr, 404 op/s 2024-03-16T10:21:11.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:11 smithi194 ceph-mon[36445]: pgmap v374: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.1 MiB/s wr, 415 op/s 2024-03-16T10:21:11.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:11 smithi039 ceph-mon[31510]: pgmap v374: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.1 MiB/s wr, 415 op/s 2024-03-16T10:21:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:11 smithi105 ceph-mon[36559]: pgmap v374: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.1 MiB/s wr, 415 op/s 2024-03-16T10:21:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 690.0 s, 470.6 tps, lat 220.331 ms stddev 272.285 2024-03-16T10:21:13.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:13 smithi039 ceph-mon[31510]: osdmap e109: 12 total, 12 up, 12 in 2024-03-16T10:21:13.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:13 smithi039 ceph-mon[31510]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 405 op/s 2024-03-16T10:21:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:13 smithi105 ceph-mon[36559]: osdmap e109: 12 total, 12 up, 12 in 2024-03-16T10:21:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:13 smithi105 ceph-mon[36559]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 405 op/s 2024-03-16T10:21:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:13 smithi194 ceph-mon[36445]: osdmap e109: 12 total, 12 up, 12 in 2024-03-16T10:21:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:13 smithi194 ceph-mon[36445]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 4.1 MiB/s wr, 405 op/s 2024-03-16T10:21:15.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:15 smithi105 ceph-mon[36559]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-03-16T10:21:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:15 smithi194 ceph-mon[36445]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-03-16T10:21:15.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:15 smithi039 ceph-mon[31510]: pgmap v377: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-03-16T10:21:17.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:17 smithi105 ceph-mon[36559]: pgmap v378: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.9 MiB/s wr, 440 op/s 2024-03-16T10:21:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:17 smithi194 ceph-mon[36445]: pgmap v378: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.9 MiB/s wr, 440 op/s 2024-03-16T10:21:17.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:17 smithi039 ceph-mon[31510]: pgmap v378: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 4.9 MiB/s wr, 440 op/s 2024-03-16T10:21:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 695.0 s, 446.9 tps, lat 225.569 ms stddev 274.674 2024-03-16T10:21:19.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:19 smithi105 ceph-mon[36559]: pgmap v379: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-03-16T10:21:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:19 smithi194 ceph-mon[36445]: pgmap v379: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-03-16T10:21:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:19 smithi039 ceph-mon[31510]: pgmap v379: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 469 op/s 2024-03-16T10:21:21.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:21 smithi105 ceph-mon[36559]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 433 op/s 2024-03-16T10:21:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:21 smithi194 ceph-mon[36445]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 433 op/s 2024-03-16T10:21:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:21 smithi039 ceph-mon[31510]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.2 MiB/s wr, 433 op/s 2024-03-16T10:21:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 700.0 s, 435.4 tps, lat 228.255 ms stddev 260.527 2024-03-16T10:21:23.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:23 smithi105 ceph-mon[36559]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2024-03-16T10:21:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:23 smithi194 ceph-mon[36445]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2024-03-16T10:21:23.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:23 smithi039 ceph-mon[31510]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2024-03-16T10:21:25.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:25 smithi194 ceph-mon[36445]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2024-03-16T10:21:25.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:25 smithi039 ceph-mon[31510]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2024-03-16T10:21:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:25 smithi105 ceph-mon[36559]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.6 MiB/s wr, 472 op/s 2024-03-16T10:21:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:27 smithi194 ceph-mon[36445]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 5.9 MiB/s wr, 484 op/s 2024-03-16T10:21:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:27 smithi039 ceph-mon[31510]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 5.9 MiB/s wr, 484 op/s 2024-03-16T10:21:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:27 smithi105 ceph-mon[36559]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 5.9 MiB/s wr, 484 op/s 2024-03-16T10:21:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 705.0 s, 418.0 tps, lat 225.077 ms stddev 230.532 2024-03-16T10:21:29.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:29 smithi194 ceph-mon[36445]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 477 op/s 2024-03-16T10:21:29.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:29 smithi039 ceph-mon[31510]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 477 op/s 2024-03-16T10:21:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:29 smithi105 ceph-mon[36559]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 477 op/s 2024-03-16T10:21:31.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:31 smithi194 ceph-mon[36445]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.3 MiB/s wr, 375 op/s 2024-03-16T10:21:31.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:31 smithi039 ceph-mon[31510]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.3 MiB/s wr, 375 op/s 2024-03-16T10:21:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:31 smithi105 ceph-mon[36559]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.3 MiB/s wr, 375 op/s 2024-03-16T10:21:33.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 710.0 s, 423.8 tps, lat 241.602 ms stddev 319.085 2024-03-16T10:21:33.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:33 smithi194 ceph-mon[36445]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-03-16T10:21:33.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:33 smithi039 ceph-mon[31510]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-03-16T10:21:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:33 smithi105 ceph-mon[36559]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-03-16T10:21:35.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:35 smithi194 ceph-mon[36445]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-03-16T10:21:35.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:35 smithi039 ceph-mon[31510]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-03-16T10:21:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:35 smithi105 ceph-mon[36559]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 5.3 MiB/s wr, 453 op/s 2024-03-16T10:21:37.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:37 smithi194 ceph-mon[36445]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.4 MiB/s wr, 466 op/s 2024-03-16T10:21:37.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:37 smithi039 ceph-mon[31510]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.4 MiB/s wr, 466 op/s 2024-03-16T10:21:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:37 smithi105 ceph-mon[36559]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.4 MiB/s wr, 466 op/s 2024-03-16T10:21:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 715.0 s, 421.8 tps, lat 235.908 ms stddev 296.939 2024-03-16T10:21:39.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:39 smithi194 ceph-mon[36445]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 464 op/s 2024-03-16T10:21:39.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:39 smithi039 ceph-mon[31510]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 464 op/s 2024-03-16T10:21:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:39 smithi105 ceph-mon[36559]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.0 MiB/s wr, 464 op/s 2024-03-16T10:21:41.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:41 smithi194 ceph-mon[36445]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 381 op/s 2024-03-16T10:21:41.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:41 smithi039 ceph-mon[31510]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 381 op/s 2024-03-16T10:21:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:41 smithi105 ceph-mon[36559]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.2 MiB/s wr, 381 op/s 2024-03-16T10:21:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 720.0 s, 460.8 tps, lat 219.594 ms stddev 275.281 2024-03-16T10:21:43.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:43 smithi039 ceph-mon[31510]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-03-16T10:21:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:43 smithi105 ceph-mon[36559]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-03-16T10:21:43.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:43 smithi194 ceph-mon[36445]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-03-16T10:21:45.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:45 smithi039 ceph-mon[31510]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-03-16T10:21:45.870 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:45 smithi105 ceph-mon[36559]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-03-16T10:21:45.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:45 smithi194 ceph-mon[36445]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 5.5 MiB/s wr, 474 op/s 2024-03-16T10:21:47.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:47 smithi039 ceph-mon[31510]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 465 op/s 2024-03-16T10:21:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:47 smithi105 ceph-mon[36559]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 465 op/s 2024-03-16T10:21:47.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:47 smithi194 ceph-mon[36445]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 465 op/s 2024-03-16T10:21:48.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 725.0 s, 462.4 tps, lat 217.340 ms stddev 289.807 2024-03-16T10:21:49.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:49 smithi039 ceph-mon[31510]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:21:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:49 smithi105 ceph-mon[36559]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:21:49.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:49 smithi194 ceph-mon[36445]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:21:50.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:50 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:50.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:50 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:51.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:50 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:51 smithi105 ceph-mon[36559]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.3 MiB/s wr, 382 op/s 2024-03-16T10:21:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:51 smithi194 ceph-mon[36445]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.3 MiB/s wr, 382 op/s 2024-03-16T10:21:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:51 smithi039 ceph-mon[31510]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.3 MiB/s wr, 382 op/s 2024-03-16T10:21:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 730.0 s, 465.4 tps, lat 218.354 ms stddev 274.703 2024-03-16T10:21:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:53 smithi105 ceph-mon[36559]: pgmap v396: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.9 MiB/s wr, 468 op/s 2024-03-16T10:21:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:53 smithi194 ceph-mon[36445]: pgmap v396: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.9 MiB/s wr, 468 op/s 2024-03-16T10:21:54.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:53 smithi039 ceph-mon[31510]: pgmap v396: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.9 MiB/s wr, 468 op/s 2024-03-16T10:21:54.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:54 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:21:54.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:54 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:21:55.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:54 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:21:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:55 smithi105 ceph-mon[36559]: pgmap v397: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-16T10:21:55.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:55 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:55 smithi194 ceph-mon[36445]: pgmap v397: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-16T10:21:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:55 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:55 smithi039 ceph-mon[31510]: pgmap v397: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 5.2 MiB/s wr, 483 op/s 2024-03-16T10:21:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:55 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:21:56.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:56 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:21:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:56 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:21:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:56 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:21:56.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:56 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:21:56.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:56 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:21:56.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:56 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:21:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:56 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:21:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:56 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:21:57.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:56 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:21:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:57 smithi105 ceph-mon[36559]: pgmap v398: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-16T10:21:57.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:57 smithi194 ceph-mon[36445]: pgmap v398: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-16T10:21:58.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:57 smithi039 ceph-mon[31510]: pgmap v398: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.8 MiB/s wr, 468 op/s 2024-03-16T10:21:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 735.0 s, 386.4 tps, lat 249.890 ms stddev 278.966 2024-03-16T10:21:59.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:21:59 smithi105 ceph-mon[36559]: pgmap v399: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:21:59.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:21:59 smithi194 ceph-mon[36445]: pgmap v399: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:22:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:21:59 smithi039 ceph-mon[31510]: pgmap v399: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-03-16T10:22:01.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:01 smithi105 ceph-mon[36559]: pgmap v400: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 3.8 MiB/s wr, 368 op/s 2024-03-16T10:22:01.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:01 smithi194 ceph-mon[36445]: pgmap v400: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 3.8 MiB/s wr, 368 op/s 2024-03-16T10:22:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:01 smithi039 ceph-mon[31510]: pgmap v400: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 3.8 MiB/s wr, 368 op/s 2024-03-16T10:22:02.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:02 smithi105 ceph-mon[36559]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-03-16T10:22:02.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:02 smithi194 ceph-mon[36445]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-03-16T10:22:03.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:02 smithi039 ceph-mon[31510]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-03-16T10:22:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 740.0 s, 441.8 tps, lat 229.006 ms stddev 296.807 2024-03-16T10:22:05.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:05 smithi194 ceph-mon[36445]: osdmap e110: 12 total, 12 up, 12 in 2024-03-16T10:22:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:05 smithi194 ceph-mon[36445]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-03-16T10:22:05.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:05 smithi039 ceph-mon[31510]: osdmap e110: 12 total, 12 up, 12 in 2024-03-16T10:22:05.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:05 smithi039 ceph-mon[31510]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-03-16T10:22:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:05 smithi105 ceph-mon[36559]: osdmap e110: 12 total, 12 up, 12 in 2024-03-16T10:22:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:05 smithi105 ceph-mon[36559]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.4 MiB/s wr, 442 op/s 2024-03-16T10:22:07.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:07 smithi105 ceph-mon[36559]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-03-16T10:22:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:07 smithi194 ceph-mon[36445]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-03-16T10:22:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:07 smithi039 ceph-mon[31510]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.9 MiB/s wr, 379 op/s 2024-03-16T10:22:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 745.0 s, 365.6 tps, lat 276.476 ms stddev 343.636 2024-03-16T10:22:09.616 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:09 smithi105 ceph-mon[36559]: pgmap v405: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:09 smithi194 ceph-mon[36445]: pgmap v405: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:09.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:09 smithi039 ceph-mon[31510]: pgmap v405: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:11.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:11 smithi105 ceph-mon[36559]: pgmap v406: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:11 smithi194 ceph-mon[36445]: pgmap v406: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:11 smithi039 ceph-mon[31510]: pgmap v406: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-03-16T10:22:12.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:12 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:12.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:12 smithi194 ceph-mon[36445]: osdmap e111: 12 total, 12 up, 12 in 2024-03-16T10:22:12.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:12 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:12.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:12 smithi039 ceph-mon[31510]: osdmap e111: 12 total, 12 up, 12 in 2024-03-16T10:22:12.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:12 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:12.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:12 smithi105 ceph-mon[36559]: osdmap e111: 12 total, 12 up, 12 in 2024-03-16T10:22:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 750.0 s, 458.2 tps, lat 224.866 ms stddev 239.066 2024-03-16T10:22:13.655 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:13 smithi039 ceph-mon[31510]: pgmap v408: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-03-16T10:22:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:13 smithi194 ceph-mon[36445]: pgmap v408: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-03-16T10:22:13.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:13 smithi105 ceph-mon[36559]: pgmap v408: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.6 MiB/s wr, 378 op/s 2024-03-16T10:22:15.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:15 smithi194 ceph-mon[36445]: pgmap v409: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-03-16T10:22:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:15 smithi039 ceph-mon[31510]: pgmap v409: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-03-16T10:22:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:15 smithi105 ceph-mon[36559]: pgmap v409: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 431 op/s 2024-03-16T10:22:17.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:17 smithi194 ceph-mon[36445]: pgmap v410: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-03-16T10:22:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:17 smithi039 ceph-mon[31510]: pgmap v410: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-03-16T10:22:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:17 smithi105 ceph-mon[36559]: pgmap v410: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 4.3 MiB/s wr, 420 op/s 2024-03-16T10:22:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 755.0 s, 314.6 tps, lat 306.183 ms stddev 478.760 2024-03-16T10:22:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:19 smithi194 ceph-mon[36445]: pgmap v411: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:19 smithi039 ceph-mon[31510]: pgmap v411: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:19 smithi105 ceph-mon[36559]: pgmap v411: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:21.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:21 smithi194 ceph-mon[36445]: pgmap v412: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:21 smithi039 ceph-mon[31510]: pgmap v412: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:21 smithi105 ceph-mon[36559]: pgmap v412: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.9 MiB/s wr, 381 op/s 2024-03-16T10:22:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 760.0 s, 452.4 tps, lat 224.389 ms stddev 247.219 2024-03-16T10:22:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:23 smithi194 ceph-mon[36445]: pgmap v413: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.3 MiB/s wr, 373 op/s 2024-03-16T10:22:23.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:23 smithi039 ceph-mon[31510]: pgmap v413: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.3 MiB/s wr, 373 op/s 2024-03-16T10:22:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:23 smithi105 ceph-mon[36559]: pgmap v413: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.3 MiB/s wr, 373 op/s 2024-03-16T10:22:25.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:25 smithi194 ceph-mon[36445]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.4 MiB/s wr, 417 op/s 2024-03-16T10:22:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:25 smithi039 ceph-mon[31510]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.4 MiB/s wr, 417 op/s 2024-03-16T10:22:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:25 smithi105 ceph-mon[36559]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.4 MiB/s wr, 417 op/s 2024-03-16T10:22:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:27 smithi194 ceph-mon[36445]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-03-16T10:22:27.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:27 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:27.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:27 smithi039 ceph-mon[31510]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-03-16T10:22:27.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:27 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:27 smithi105 ceph-mon[36559]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-03-16T10:22:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:27 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:22:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 765.0 s, 430.0 tps, lat 235.060 ms stddev 282.103 2024-03-16T10:22:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:29 smithi039 ceph-mon[31510]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-03-16T10:22:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:29 smithi105 ceph-mon[36559]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-03-16T10:22:29.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:29 smithi194 ceph-mon[36445]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-03-16T10:22:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:31 smithi039 ceph-mon[31510]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-03-16T10:22:31.757 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:31 smithi194 ceph-mon[36445]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-03-16T10:22:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:31 smithi105 ceph-mon[36559]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-03-16T10:22:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 770.0 s, 390.2 tps, lat 240.087 ms stddev 259.797 2024-03-16T10:22:33.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:33 smithi039 ceph-mon[31510]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-03-16T10:22:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:33 smithi105 ceph-mon[36559]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-03-16T10:22:33.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:33 smithi194 ceph-mon[36445]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-03-16T10:22:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:35 smithi039 ceph-mon[31510]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-16T10:22:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:35 smithi105 ceph-mon[36559]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-16T10:22:35.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:35 smithi194 ceph-mon[36445]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-03-16T10:22:37.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:37 smithi039 ceph-mon[31510]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-03-16T10:22:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:37 smithi105 ceph-mon[36559]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-03-16T10:22:37.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:37 smithi194 ceph-mon[36445]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 3.7 MiB/s wr, 374 op/s 2024-03-16T10:22:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 775.0 s, 412.8 tps, lat 245.597 ms stddev 340.233 2024-03-16T10:22:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:39 smithi105 ceph-mon[36559]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-03-16T10:22:39.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:39 smithi194 ceph-mon[36445]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-03-16T10:22:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:39 smithi039 ceph-mon[31510]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-03-16T10:22:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:41 smithi105 ceph-mon[36559]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-03-16T10:22:41.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:41 smithi194 ceph-mon[36445]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-03-16T10:22:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:41 smithi039 ceph-mon[31510]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-03-16T10:22:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 780.0 s, 414.0 tps, lat 246.806 ms stddev 342.884 2024-03-16T10:22:43.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:43 smithi039 ceph-mon[31510]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.3 MiB/s wr, 424 op/s 2024-03-16T10:22:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:43 smithi105 ceph-mon[36559]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.3 MiB/s wr, 424 op/s 2024-03-16T10:22:43.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:43 smithi194 ceph-mon[36445]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 4.3 MiB/s wr, 424 op/s 2024-03-16T10:22:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:45 smithi105 ceph-mon[36559]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-03-16T10:22:45.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:45 smithi194 ceph-mon[36445]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-03-16T10:22:46.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:45 smithi039 ceph-mon[31510]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-03-16T10:22:47.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:47 smithi105 ceph-mon[36559]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-03-16T10:22:48.130 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:47 smithi039 ceph-mon[31510]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-03-16T10:22:48.130 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:47 smithi194 ceph-mon[36445]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 3.7 MiB/s wr, 373 op/s 2024-03-16T10:22:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 785.0 s, 398.8 tps, lat 250.768 ms stddev 280.234 2024-03-16T10:22:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:49 smithi105 ceph-mon[36559]: pgmap v426: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-03-16T10:22:49.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:49 smithi194 ceph-mon[36445]: pgmap v426: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-03-16T10:22:50.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:49 smithi039 ceph-mon[31510]: pgmap v426: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-03-16T10:22:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:51 smithi105 ceph-mon[36559]: pgmap v427: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.7 MiB/s wr, 371 op/s 2024-03-16T10:22:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:51 smithi194 ceph-mon[36445]: pgmap v427: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.7 MiB/s wr, 371 op/s 2024-03-16T10:22:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:51 smithi039 ceph-mon[31510]: pgmap v427: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 3.7 MiB/s wr, 371 op/s 2024-03-16T10:22:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 790.0 s, 402.0 tps, lat 246.772 ms stddev 331.430 2024-03-16T10:22:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:53 smithi105 ceph-mon[36559]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-03-16T10:22:53.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:53 smithi194 ceph-mon[36445]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-03-16T10:22:54.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:53 smithi039 ceph-mon[31510]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 4.5 MiB/s wr, 441 op/s 2024-03-16T10:22:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:55 smithi105 ceph-mon[36559]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 5.3 MiB/s wr, 484 op/s 2024-03-16T10:22:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:55 smithi194 ceph-mon[36445]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 5.3 MiB/s wr, 484 op/s 2024-03-16T10:22:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:55 smithi039 ceph-mon[31510]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 5.3 MiB/s wr, 484 op/s 2024-03-16T10:22:56.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:56 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:22:56.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:56 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:22:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:56 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:22:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:57 smithi105 ceph-mon[36559]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-03-16T10:22:57.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:57 smithi194 ceph-mon[36445]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-03-16T10:22:58.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:57 smithi039 ceph-mon[31510]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 397 op/s 2024-03-16T10:22:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 795.0 s, 453.2 tps, lat 228.363 ms stddev 254.808 2024-03-16T10:22:58.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:58 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:22:58.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:58 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:22:58.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:58 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:22:58.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:58 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:22:58.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:58 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:22:58.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:58 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:22:59.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:58 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:22:59.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:58 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:22:59.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:58 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:22:59.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:22:59 smithi105 ceph-mon[36559]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 523 op/s 2024-03-16T10:22:59.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:22:59 smithi194 ceph-mon[36445]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 523 op/s 2024-03-16T10:23:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:22:59 smithi039 ceph-mon[31510]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 523 op/s 2024-03-16T10:23:01.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:01 smithi194 ceph-mon[36445]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.3 MiB/s wr, 380 op/s 2024-03-16T10:23:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:01 smithi039 ceph-mon[31510]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.3 MiB/s wr, 380 op/s 2024-03-16T10:23:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:01 smithi105 ceph-mon[36559]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 4.3 MiB/s wr, 380 op/s 2024-03-16T10:23:02.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:02 smithi194 ceph-mon[36445]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.4 MiB/s wr, 398 op/s 2024-03-16T10:23:03.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:02 smithi039 ceph-mon[31510]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.4 MiB/s wr, 398 op/s 2024-03-16T10:23:03.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:02 smithi105 ceph-mon[36559]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.4 MiB/s wr, 398 op/s 2024-03-16T10:23:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 800.0 s, 347.2 tps, lat 263.208 ms stddev 346.416 2024-03-16T10:23:05.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:05 smithi194 ceph-mon[36445]: osdmap e112: 12 total, 12 up, 12 in 2024-03-16T10:23:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:05 smithi194 ceph-mon[36445]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-03-16T10:23:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:05 smithi039 ceph-mon[31510]: osdmap e112: 12 total, 12 up, 12 in 2024-03-16T10:23:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:05 smithi039 ceph-mon[31510]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-03-16T10:23:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:05 smithi105 ceph-mon[36559]: osdmap e112: 12 total, 12 up, 12 in 2024-03-16T10:23:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:05 smithi105 ceph-mon[36559]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.0 MiB/s wr, 409 op/s 2024-03-16T10:23:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:07 smithi194 ceph-mon[36445]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-16T10:23:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:07 smithi039 ceph-mon[31510]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-16T10:23:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:07 smithi105 ceph-mon[36559]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.4 MiB/s wr, 390 op/s 2024-03-16T10:23:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 805.0 s, 397.2 tps, lat 271.245 ms stddev 332.103 2024-03-16T10:23:09.666 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:09 smithi105 ceph-mon[36559]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-03-16T10:23:09.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:09 smithi194 ceph-mon[36445]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-03-16T10:23:09.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:09 smithi039 ceph-mon[31510]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.4 MiB/s wr, 365 op/s 2024-03-16T10:23:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:11 smithi194 ceph-mon[36445]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-03-16T10:23:11.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:11 smithi039 ceph-mon[31510]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-03-16T10:23:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:11 smithi105 ceph-mon[36559]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-03-16T10:23:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 810.0 s, 407.4 tps, lat 241.891 ms stddev 269.629 2024-03-16T10:23:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:13 smithi039 ceph-mon[31510]: osdmap e113: 12 total, 12 up, 12 in 2024-03-16T10:23:13.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:13 smithi039 ceph-mon[31510]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-03-16T10:23:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:13 smithi105 ceph-mon[36559]: osdmap e113: 12 total, 12 up, 12 in 2024-03-16T10:23:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:13 smithi105 ceph-mon[36559]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-03-16T10:23:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:13 smithi194 ceph-mon[36445]: osdmap e113: 12 total, 12 up, 12 in 2024-03-16T10:23:13.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:13 smithi194 ceph-mon[36445]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.5 MiB/s wr, 278 op/s 2024-03-16T10:23:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:15 smithi194 ceph-mon[36445]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-16T10:23:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:15 smithi039 ceph-mon[31510]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-16T10:23:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:15 smithi105 ceph-mon[36559]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-03-16T10:23:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:17 smithi194 ceph-mon[36445]: pgmap v442: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-03-16T10:23:17.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:17 smithi039 ceph-mon[31510]: pgmap v442: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-03-16T10:23:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:17 smithi105 ceph-mon[36559]: pgmap v442: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-03-16T10:23:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 815.0 s, 432.2 tps, lat 230.072 ms stddev 259.116 2024-03-16T10:23:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:19 smithi194 ceph-mon[36445]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-03-16T10:23:19.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:19 smithi039 ceph-mon[31510]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-03-16T10:23:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:19 smithi105 ceph-mon[36559]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.1 MiB/s wr, 443 op/s 2024-03-16T10:23:21.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:21 smithi194 ceph-mon[36445]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 455 op/s 2024-03-16T10:23:21.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:21 smithi039 ceph-mon[31510]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 455 op/s 2024-03-16T10:23:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:21 smithi105 ceph-mon[36559]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.3 MiB/s wr, 455 op/s 2024-03-16T10:23:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 820.0 s, 443.6 tps, lat 231.762 ms stddev 262.104 2024-03-16T10:23:23.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:23 smithi194 ceph-mon[36445]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:23:23.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:23 smithi039 ceph-mon[31510]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:23:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:23 smithi105 ceph-mon[36559]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.2 MiB/s wr, 448 op/s 2024-03-16T10:23:25.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:25 smithi194 ceph-mon[36445]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-03-16T10:23:25.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:25 smithi039 ceph-mon[31510]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-03-16T10:23:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:25 smithi105 ceph-mon[36559]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-03-16T10:23:27.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:27 smithi194 ceph-mon[36445]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-16T10:23:27.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:27 smithi039 ceph-mon[31510]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-16T10:23:27.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:27 smithi105 ceph-mon[36559]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-03-16T10:23:28.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 825.0 s, 457.4 tps, lat 211.861 ms stddev 240.588 2024-03-16T10:23:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:29 smithi039 ceph-mon[31510]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-03-16T10:23:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:29 smithi105 ceph-mon[36559]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-03-16T10:23:29.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:29 smithi194 ceph-mon[36445]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.9 MiB/s wr, 522 op/s 2024-03-16T10:23:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:31 smithi039 ceph-mon[31510]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-03-16T10:23:31.757 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:31 smithi194 ceph-mon[36445]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-03-16T10:23:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:31 smithi105 ceph-mon[36559]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-03-16T10:23:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 830.0 s, 506.6 tps, lat 200.670 ms stddev 239.428 2024-03-16T10:23:33.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:33 smithi039 ceph-mon[31510]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-03-16T10:23:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:33 smithi105 ceph-mon[36559]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-03-16T10:23:33.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:33 smithi194 ceph-mon[36445]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-03-16T10:23:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:35 smithi105 ceph-mon[36559]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 523 op/s 2024-03-16T10:23:35.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:35 smithi194 ceph-mon[36445]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 523 op/s 2024-03-16T10:23:36.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:35 smithi039 ceph-mon[31510]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 523 op/s 2024-03-16T10:23:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:37 smithi105 ceph-mon[36559]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-03-16T10:23:37.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:37 smithi194 ceph-mon[36445]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-03-16T10:23:38.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:37 smithi039 ceph-mon[31510]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 4.0 MiB/s wr, 407 op/s 2024-03-16T10:23:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 835.0 s, 433.4 tps, lat 225.684 ms stddev 289.634 2024-03-16T10:23:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:39 smithi105 ceph-mon[36559]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-03-16T10:23:39.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:39 smithi194 ceph-mon[36445]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-03-16T10:23:40.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:39 smithi039 ceph-mon[31510]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-03-16T10:23:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:41 smithi105 ceph-mon[36559]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.7 MiB/s wr, 397 op/s 2024-03-16T10:23:41.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:41 smithi194 ceph-mon[36445]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.7 MiB/s wr, 397 op/s 2024-03-16T10:23:42.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:41 smithi039 ceph-mon[31510]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.7 MiB/s wr, 397 op/s 2024-03-16T10:23:43.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 840.0 s, 412.0 tps, lat 233.716 ms stddev 280.675 2024-03-16T10:23:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:43 smithi105 ceph-mon[36559]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-03-16T10:23:43.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:43 smithi194 ceph-mon[36445]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-03-16T10:23:44.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:43 smithi039 ceph-mon[31510]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 3.9 MiB/s wr, 422 op/s 2024-03-16T10:23:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:45 smithi105 ceph-mon[36559]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-03-16T10:23:45.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:45 smithi194 ceph-mon[36445]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-03-16T10:23:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:45 smithi039 ceph-mon[31510]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.1 MiB/s wr, 536 op/s 2024-03-16T10:23:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:47 smithi105 ceph-mon[36559]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-03-16T10:23:47.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:47 smithi194 ceph-mon[36445]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-03-16T10:23:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:47 smithi039 ceph-mon[31510]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-03-16T10:23:48.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 845.0 s, 438.2 tps, lat 235.319 ms stddev 328.242 2024-03-16T10:23:49.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:49 smithi105 ceph-mon[36559]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-16T10:23:49.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:49 smithi194 ceph-mon[36445]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-16T10:23:50.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:49 smithi039 ceph-mon[31510]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-03-16T10:23:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:51 smithi105 ceph-mon[36559]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-03-16T10:23:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:51 smithi194 ceph-mon[36445]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-03-16T10:23:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:51 smithi039 ceph-mon[31510]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 3.5 MiB/s wr, 400 op/s 2024-03-16T10:23:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 850.0 s, 396.2 tps, lat 259.595 ms stddev 341.199 2024-03-16T10:23:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:53 smithi105 ceph-mon[36559]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-03-16T10:23:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:53 smithi194 ceph-mon[36445]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-03-16T10:23:54.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:53 smithi039 ceph-mon[31510]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 3.9 MiB/s wr, 426 op/s 2024-03-16T10:23:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:55 smithi194 ceph-mon[36445]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-03-16T10:23:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:55 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:23:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:55 smithi039 ceph-mon[31510]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-03-16T10:23:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:55 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:23:56.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:55 smithi105 ceph-mon[36559]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.8 MiB/s wr, 527 op/s 2024-03-16T10:23:56.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:55 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:23:57.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:57 smithi194 ceph-mon[36445]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.5 MiB/s wr, 386 op/s 2024-03-16T10:23:58.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:57 smithi039 ceph-mon[31510]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.5 MiB/s wr, 386 op/s 2024-03-16T10:23:58.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:57 smithi105 ceph-mon[36559]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 3.5 MiB/s wr, 386 op/s 2024-03-16T10:23:58.489 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 855.0 s, 402.6 tps, lat 243.474 ms stddev 270.142 2024-03-16T10:23:58.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:58 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:23:59.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:58 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:23:59.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:58 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:23:59.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:23:59 smithi194 ceph-mon[36445]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-16T10:24:00.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:23:59 smithi039 ceph-mon[31510]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-16T10:24:00.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:23:59 smithi105 ceph-mon[36559]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-03-16T10:24:00.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:00 smithi194 ceph-mon[36445]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-16T10:24:01.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:00 smithi039 ceph-mon[31510]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-16T10:24:01.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:00 smithi105 ceph-mon[36559]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-03-16T10:24:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:01 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:24:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:01 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:24:01.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:01 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:24:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:01 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:24:02.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:01 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:24:02.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:01 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:24:02.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:01 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:24:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:01 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:24:02.115 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:01 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:24:02.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:02 smithi194 ceph-mon[36445]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-03-16T10:24:03.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:02 smithi039 ceph-mon[31510]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-03-16T10:24:03.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:02 smithi105 ceph-mon[36559]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.3 MiB/s wr, 354 op/s 2024-03-16T10:24:03.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 860.0 s, 382.0 tps, lat 258.105 ms stddev 328.143 2024-03-16T10:24:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:05 smithi194 ceph-mon[36445]: osdmap e114: 12 total, 12 up, 12 in 2024-03-16T10:24:05.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:05 smithi194 ceph-mon[36445]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-03-16T10:24:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:05 smithi039 ceph-mon[31510]: osdmap e114: 12 total, 12 up, 12 in 2024-03-16T10:24:05.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:05 smithi039 ceph-mon[31510]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-03-16T10:24:05.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:05 smithi105 ceph-mon[36559]: osdmap e114: 12 total, 12 up, 12 in 2024-03-16T10:24:05.615 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:05 smithi105 ceph-mon[36559]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-03-16T10:24:07.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:07 smithi194 ceph-mon[36445]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-03-16T10:24:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:07 smithi039 ceph-mon[31510]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-03-16T10:24:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:07 smithi105 ceph-mon[36559]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-03-16T10:24:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 865.0 s, 273.4 tps, lat 375.393 ms stddev 531.097 2024-03-16T10:24:09.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:09 smithi194 ceph-mon[36445]: pgmap v469: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-16T10:24:09.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:09 smithi039 ceph-mon[31510]: pgmap v469: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-16T10:24:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:09 smithi105 ceph-mon[36559]: pgmap v469: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 3.1 MiB/s wr, 338 op/s 2024-03-16T10:24:11.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:11 smithi194 ceph-mon[36445]: pgmap v470: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-16T10:24:11.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:11 smithi039 ceph-mon[31510]: pgmap v470: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-16T10:24:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:11 smithi105 ceph-mon[36559]: pgmap v470: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-03-16T10:24:12.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:12 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:12.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:12 smithi194 ceph-mon[36445]: osdmap e115: 12 total, 12 up, 12 in 2024-03-16T10:24:12.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:12 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:12.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:12 smithi039 ceph-mon[31510]: osdmap e115: 12 total, 12 up, 12 in 2024-03-16T10:24:12.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:12 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:12.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:12 smithi105 ceph-mon[36559]: osdmap e115: 12 total, 12 up, 12 in 2024-03-16T10:24:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 870.0 s, 339.0 tps, lat 282.879 ms stddev 331.890 2024-03-16T10:24:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:13 smithi194 ceph-mon[36445]: pgmap v472: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-03-16T10:24:13.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:13 smithi039 ceph-mon[31510]: pgmap v472: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-03-16T10:24:13.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:13 smithi105 ceph-mon[36559]: pgmap v472: 129 pgs: 129 active+clean; 2.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-03-16T10:24:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:15 smithi194 ceph-mon[36445]: pgmap v473: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-03-16T10:24:15.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:15 smithi039 ceph-mon[31510]: pgmap v473: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-03-16T10:24:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:15 smithi105 ceph-mon[36559]: pgmap v473: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-03-16T10:24:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:17 smithi039 ceph-mon[31510]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-16T10:24:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:17 smithi105 ceph-mon[36559]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-16T10:24:17.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:17 smithi194 ceph-mon[36445]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.0 MiB/s wr, 339 op/s 2024-03-16T10:24:18.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 875.0 s, 388.4 tps, lat 264.346 ms stddev 332.649 2024-03-16T10:24:19.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:19 smithi039 ceph-mon[31510]: pgmap v475: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-03-16T10:24:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:19 smithi105 ceph-mon[36559]: pgmap v475: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-03-16T10:24:19.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:19 smithi194 ceph-mon[36445]: pgmap v475: 129 pgs: 129 active+clean; 2.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-03-16T10:24:20.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:20 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:20.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:20 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:20.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:20 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:21.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:21 smithi039 ceph-mon[31510]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-03-16T10:24:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:21 smithi105 ceph-mon[36559]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-03-16T10:24:21.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:21 smithi194 ceph-mon[36445]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-03-16T10:24:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 880.0 s, 469.2 tps, lat 218.602 ms stddev 298.199 2024-03-16T10:24:23.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:23 smithi039 ceph-mon[31510]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-03-16T10:24:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:23 smithi105 ceph-mon[36559]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-03-16T10:24:23.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:23 smithi194 ceph-mon[36445]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.9 MiB/s wr, 432 op/s 2024-03-16T10:24:25.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:25 smithi105 ceph-mon[36559]: pgmap v478: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-03-16T10:24:25.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:25 smithi194 ceph-mon[36445]: pgmap v478: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-03-16T10:24:26.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:25 smithi039 ceph-mon[31510]: pgmap v478: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-03-16T10:24:27.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:27 smithi105 ceph-mon[36559]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-03-16T10:24:27.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:27 smithi194 ceph-mon[36445]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-03-16T10:24:28.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:27 smithi039 ceph-mon[31510]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.4 MiB/s wr, 407 op/s 2024-03-16T10:24:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 885.0 s, 404.8 tps, lat 242.428 ms stddev 268.244 2024-03-16T10:24:29.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:29 smithi105 ceph-mon[36559]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-03-16T10:24:29.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:29 smithi194 ceph-mon[36445]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-03-16T10:24:30.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:29 smithi039 ceph-mon[31510]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-03-16T10:24:31.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:31 smithi105 ceph-mon[36559]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-03-16T10:24:31.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:31 smithi194 ceph-mon[36445]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-03-16T10:24:32.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:31 smithi039 ceph-mon[31510]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-03-16T10:24:32.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:32 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:32.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:32 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:33.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:32 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:33.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 890.0 s, 393.4 tps, lat 252.361 ms stddev 300.397 2024-03-16T10:24:33.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:33 smithi105 ceph-mon[36559]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-16T10:24:33.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:33 smithi194 ceph-mon[36445]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-16T10:24:34.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:33 smithi039 ceph-mon[31510]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-03-16T10:24:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:35 smithi105 ceph-mon[36559]: pgmap v483: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-03-16T10:24:35.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:35 smithi194 ceph-mon[36445]: pgmap v483: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-03-16T10:24:36.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:35 smithi039 ceph-mon[31510]: pgmap v483: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 4.7 MiB/s wr, 499 op/s 2024-03-16T10:24:37.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:37 smithi105 ceph-mon[36559]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-03-16T10:24:37.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:37 smithi194 ceph-mon[36445]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-03-16T10:24:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:37 smithi039 ceph-mon[31510]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 4.1 MiB/s wr, 433 op/s 2024-03-16T10:24:38.487 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 895.0 s, 358.6 tps, lat 277.103 ms stddev 322.625 2024-03-16T10:24:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:39 smithi105 ceph-mon[36559]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.8 MiB/s wr, 446 op/s 2024-03-16T10:24:39.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:39 smithi194 ceph-mon[36445]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.8 MiB/s wr, 446 op/s 2024-03-16T10:24:40.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:39 smithi039 ceph-mon[31510]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 4.8 MiB/s wr, 446 op/s 2024-03-16T10:24:41.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:41 smithi105 ceph-mon[36559]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.0 MiB/s wr, 455 op/s 2024-03-16T10:24:41.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:41 smithi194 ceph-mon[36445]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.0 MiB/s wr, 455 op/s 2024-03-16T10:24:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:41 smithi039 ceph-mon[31510]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.0 MiB/s wr, 455 op/s 2024-03-16T10:24:43.488 INFO:teuthology.orchestra.run.smithi039.stderr:progress: 900.0 s, 403.3 tps, lat 250.483 ms stddev 268.944 2024-03-16T10:24:43.739 INFO:teuthology.orchestra.run.smithi039.stdout:transaction type: 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:scaling factor: 32 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:query mode: simple 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:number of clients: 100 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:number of threads: 4 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:duration: 900 s 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:number of transactions actually processed: 582268 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:latency average = 154.534 ms 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:latency stddev = 242.699 ms 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 646.782853 (including connections establishing) 2024-03-16T10:24:43.740 INFO:teuthology.orchestra.run.smithi039.stdout:tps = 646.788985 (excluding connections establishing) 2024-03-16T10:24:43.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-03-16T10:24:43.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:43 smithi194 ceph-mon[36445]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 381 op/s 2024-03-16T10:24:43.967 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:43 smithi039 ceph-mon[31510]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 381 op/s 2024-03-16T10:24:44.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:43 smithi105 ceph-mon[36559]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.2 MiB/s wr, 381 op/s 2024-03-16T10:24:45.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:45 smithi194 ceph-mon[36445]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2024-03-16T10:24:46.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:45 smithi039 ceph-mon[31510]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2024-03-16T10:24:46.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:45 smithi105 ceph-mon[36559]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2024-03-16T10:24:47.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:47 smithi194 ceph-mon[36445]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-03-16T10:24:48.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:47 smithi039 ceph-mon[31510]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-03-16T10:24:48.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:47 smithi105 ceph-mon[36559]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.0 MiB/s wr, 425 op/s 2024-03-16T10:24:48.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:48 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:49.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:48 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:49.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:48 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:49.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:49 smithi194 ceph-mon[36445]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.2 MiB/s wr, 545 op/s 2024-03-16T10:24:50.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:49 smithi039 ceph-mon[31510]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.2 MiB/s wr, 545 op/s 2024-03-16T10:24:50.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:49 smithi105 ceph-mon[36559]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 7.2 MiB/s wr, 545 op/s 2024-03-16T10:24:50.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:50 smithi194 ceph-mon[36445]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-03-16T10:24:51.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:50 smithi039 ceph-mon[31510]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-03-16T10:24:51.114 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:50 smithi105 ceph-mon[36559]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-03-16T10:24:53.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:53 smithi194 ceph-mon[36445]: pgmap v492: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 494 op/s 2024-03-16T10:24:53.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:53 smithi039 ceph-mon[31510]: pgmap v492: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 494 op/s 2024-03-16T10:24:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:53 smithi105 ceph-mon[36559]: pgmap v492: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 494 op/s 2024-03-16T10:24:54.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:54 smithi194 ceph-mon[36445]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:54.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:54 smithi039 ceph-mon[31510]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:54.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:54 smithi105 ceph-mon[36559]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-16T10:24:55.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:55 smithi039 ceph-mon[31510]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.4 MiB/s wr, 505 op/s 2024-03-16T10:24:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:55 smithi105 ceph-mon[36559]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.4 MiB/s wr, 505 op/s 2024-03-16T10:24:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:55 smithi194 ceph-mon[36445]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 5.4 MiB/s wr, 505 op/s 2024-03-16T10:24:57.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:57 smithi039 ceph-mon[31510]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-03-16T10:24:57.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:57 smithi105 ceph-mon[36559]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-03-16T10:24:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:57 smithi194 ceph-mon[36445]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 5.7 MiB/s wr, 600 op/s 2024-03-16T10:24:59.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-16T10:24:59.738 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:24:59 smithi039 ceph-mon[31510]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-16T10:24:59.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:24:59 smithi105 ceph-mon[36559]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-16T10:24:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/: 2024-03-16T10:24:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:24:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:24:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:24:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 data 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data: 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:total 36 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 . 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 .. 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 base 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Mar 16 10:09 current_logfiles 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 16 10:09 global 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 log 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_commit_ts 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_dynshmem 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 16 10:09 pg_hba.conf 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 16 10:09 pg_ident.conf 2024-03-16T10:24:59.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 pg_logical 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 pg_multixact 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_notify 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_replslot 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_serial 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_snapshots 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 pg_stat 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:24 pg_stat_tmp 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:24 pg_subtrans 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_tblspc 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pg_twophase 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Mar 16 10:24 pg_wal 2024-03-16T10:24:59.881 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 pg_xact 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 16 10:09 postgresql.auto.conf 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 16 10:09 postgresql.conf 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Mar 16 10:09 postmaster.opts 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base: 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 . 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 1 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 13493 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 16 10:16 13494 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 pgsql_tmp 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:24:59.882 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:total 7777 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 . 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 .. 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:24:59.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:09 1259 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:24:59.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:24:59.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:24:59.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:24:59.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:24:59.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:09 2619 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2619_fsm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:24:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:24:59.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:24:59.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2696 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:24:59.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:24:59.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:24:59.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:24:59.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:24:59.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:24:59.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:24:59.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:24:59.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:24:59.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:24:59.931 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:24:59.932 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:24:59.933 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:total 7777 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 16 10:09 . 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 .. 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:24:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:09 1259 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:24:59.935 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:24:59.936 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:24:59.937 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:24:59.938 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:24:59.939 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:24:59.940 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:24:59.941 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:09 2619 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2619_fsm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:24:59.942 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:24:59.943 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:24:59.944 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2696 2024-03-16T10:24:59.945 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:24:59.946 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:24:59.947 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:24:59.948 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:24:59.949 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:24:59.950 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:24:59.951 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:24:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:24:59.953 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:24:59.954 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:24:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:24:59.956 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:24:59.957 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:24:59.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:24:59 smithi194 ceph-mon[36445]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:25:00.057 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:25:00.058 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:total 541390 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 16 10:16 . 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 .. 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 112 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 113 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 1247 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1247_fsm 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1247_vm 2024-03-16T10:25:00.059 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 16 10:09 1249 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1249_fsm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1249_vm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 16 10:09 1255 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1255_fsm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1255_vm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 16 10:24 1259 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1259_fsm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1259_vm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 16 10:09 13344 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13344_fsm 2024-03-16T10:25:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13344_vm 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13346 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13348 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13349_fsm 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13349_vm 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13351 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13353 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13354_fsm 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13354_vm 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13356 2024-03-16T10:25:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13358 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 13359_fsm 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13359_vm 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 13361 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 13363 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1417 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 1418 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439558144 Mar 16 10:24 16396 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 16 10:20 16396_fsm 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:23 16396_vm 2024-03-16T10:25:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1630208 Mar 16 10:24 16397 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:24 16397_fsm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:24 16397_vm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1302528 Mar 16 10:24 16399 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:24 16399_fsm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:24 16399_vm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:24 16400 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:24 16402 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Mar 16 10:24 16404 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 31539200 Mar 16 10:24 16406 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:24 16406_fsm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:23 16406_vm 2024-03-16T10:25:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 174 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 175 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2187 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2224 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2328 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2336 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2337 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2579 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2600 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2600_fsm 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2600_vm 2024-03-16T10:25:00.064 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2601_fsm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2601_vm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2602 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2602_fsm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2602_vm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2603 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2603_fsm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2603_vm 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2604 2024-03-16T10:25:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2605 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2605_fsm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2605_vm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2606_fsm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2606_vm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2607 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2607_fsm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2607_vm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 16 10:09 2608 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2608_fsm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2608_vm 2024-03-16T10:25:00.066 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 16 10:09 2609 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2609_fsm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2609_vm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2610 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2610_fsm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2610_vm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2611 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2612_fsm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2612_vm 2024-03-16T10:25:00.067 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2613 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2615_fsm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2615_vm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2616 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2616_fsm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2616_vm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 16 10:09 2617 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2617_fsm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2617_vm 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 16 10:09 2618 2024-03-16T10:25:00.068 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2618_fsm 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2618_vm 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Mar 16 10:24 2619 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:11 2619_fsm 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2619_vm 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2620 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2650 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2651 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2652 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2653 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2654 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2655 2024-03-16T10:25:00.069 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2656 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2657 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 16 10:09 2658 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 16 10:09 2659 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2660 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2661 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2662 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2663 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2664 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2665 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2666 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2667 2024-03-16T10:25:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2668 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2669 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2670 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 16 10:09 2673 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 16 10:09 2674 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 16 10:09 2675 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2678 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2679 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2680 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2681 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2682 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2683 2024-03-16T10:25:00.071 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2684 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2685 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2686 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2687 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2688 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 16 10:09 2689 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 16 10:09 2690 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 16 10:09 2691 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2692 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2693 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:13 2696 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2699 2024-03-16T10:25:00.072 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2701 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2702 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2703 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 2704 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2753 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2753_fsm 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2753_vm 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2754 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2755 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2756 2024-03-16T10:25:00.073 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 2757 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2830 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2831 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2832 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2833 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2834 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2835 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2836 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2837 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 16 10:09 2838 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2838_fsm 2024-03-16T10:25:00.074 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2838_vm 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2839 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2840_fsm 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2840_vm 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2841 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2995 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2996 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3079_fsm 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3079_vm 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3080 2024-03-16T10:25:00.075 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3081 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3085 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3118 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3119 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 16 10:09 3164 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3256 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3257 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3258 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3350 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3351 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3379 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3380 2024-03-16T10:25:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3381 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3394 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3394_fsm 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3394_vm 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3395 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3429 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3430 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3431 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3433 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3439 2024-03-16T10:25:00.077 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3440 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 16 10:09 3455 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 16 10:09 3456 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3456_fsm 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3456_vm 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3466 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3467 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3468 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3501 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3502 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3503 2024-03-16T10:25:00.078 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3534 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3541_fsm 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3541_vm 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3542 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3574 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3575 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3576 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3596 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3597 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3598 2024-03-16T10:25:00.079 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3599 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3600_fsm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3600_vm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3601_fsm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3601_vm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3602_fsm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3602_vm 2024-03-16T10:25:00.080 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3603_fsm 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3603_vm 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3604 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3605 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3606 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3607 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3608 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 16 10:09 3609 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3712 2024-03-16T10:25:00.081 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 3764_fsm 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3764_vm 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3766 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 3767 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3997 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4143 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4144 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4145 2024-03-16T10:25:00.082 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4146 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4147 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4148 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4149 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4150 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4151 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4152 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4153 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4154 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4155 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4156 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4157 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4158 2024-03-16T10:25:00.083 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4159 2024-03-16T10:25:00.084 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4160 2024-03-16T10:25:00.084 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4161 2024-03-16T10:25:00.084 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4162 2024-03-16T10:25:00.084 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4163 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4164 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4165 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4166 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4167 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4168 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4169 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4170 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4171 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4172 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4173 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4174 2024-03-16T10:25:00.085 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 5002 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 548 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 549 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6102 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6104 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6106 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6110 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6111 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6112 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6113 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6117 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 826 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 827 2024-03-16T10:25:00.086 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 828 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Mar 16 10:09 pg_internal.init 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:09 PG_VERSION 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 16 10:09 .. 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.087 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/global: 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:total 560 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 16 10:09 . 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1213 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1213_fsm 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1213_vm 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1214 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1214_fsm 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1214_vm 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 1232 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 1233 2024-03-16T10:25:00.088 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1260 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1260_fsm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1260_vm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1261 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1261_fsm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1261_vm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1262 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 1262_fsm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 1262_vm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2396 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 16 10:09 2396_fsm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2396_vm 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2397 2024-03-16T10:25:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2671 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2672 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2676 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2677 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2694 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2695 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2697 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 16 10:09 2698 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2846 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2847 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2964 2024-03-16T10:25:00.090 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2965 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 2966 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 2967 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 3592 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 3593 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4060 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4061 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4175 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4176 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4177 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4178 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4181 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4182 2024-03-16T10:25:00.091 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4183 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4184 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 4185 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 4186 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6000 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6001 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6002 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 6100 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6114 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 6115 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:24 pg_control 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 16 10:09 pg_filenode.map 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Mar 16 10:09 pg_internal.init 2024-03-16T10:25:00.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/log: 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:total 2 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Mar 16 10:24 postgresql-Sat.log 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.093 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:total 1 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 . 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 mappings 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 16 10:24 replorigin_checkpoint 2024-03-16T10:25:00.094 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 snapshots 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 .. 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 .. 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-16T10:25:00.095 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 . 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 members 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 offsets 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 .. 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:09 0000 2024-03-16T10:25:00.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:total 8 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 16 10:09 .. 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 16 10:14 0000 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-16T10:25:00.097 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.098 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-16T10:25:00.099 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:total 15 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 16 10:24 . 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Mar 16 10:24 db_0.stat 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Mar 16 10:24 db_13494.stat 2024-03-16T10:25:00.100 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Mar 16 10:24 global.stat 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:24 . 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:total 232 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:24 . 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Mar 16 10:24 0008 2024-03-16T10:25:00.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.102 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:total 884736 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Mar 16 10:24 . 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:24 00000001000000000000005D 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:14 00000001000000000000005E 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:14 00000001000000000000005F 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:18 000000010000000000000060 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000061 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000062 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000063 2024-03-16T10:25:00.103 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:17 000000010000000000000064 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:16 000000010000000000000065 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000066 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:17 000000010000000000000067 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:16 000000010000000000000068 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:18 000000010000000000000069 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 00000001000000000000006A 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:14 00000001000000000000006B 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 00000001000000000000006C 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 00000001000000000000006D 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:16 00000001000000000000006E 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 00000001000000000000006F 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000070 2024-03-16T10:25:00.104 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:16 000000010000000000000071 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:17 000000010000000000000072 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:14 000000010000000000000073 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:18 000000010000000000000074 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:16 000000010000000000000075 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000076 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:15 000000010000000000000077 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 000000010000000000000078 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:22 000000010000000000000079 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 00000001000000000000007A 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 00000001000000000000007B 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:23 00000001000000000000007C 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:24 00000001000000000000007D 2024-03-16T10:25:00.105 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 00000001000000000000007E 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 00000001000000000000007F 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 000000010000000000000080 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 000000010000000000000081 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 000000010000000000000082 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:22 000000010000000000000083 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 000000010000000000000084 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 000000010000000000000085 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 000000010000000000000086 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:23 000000010000000000000087 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 000000010000000000000088 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:23 000000010000000000000089 2024-03-16T10:25:00.106 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:19 00000001000000000000008A 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 00000001000000000000008B 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 00000001000000000000008C 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 00000001000000000000008D 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:21 00000001000000000000008E 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:22 00000001000000000000008F 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 000000010000000000000090 2024-03-16T10:25:00.107 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 000000010000000000000091 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 16 10:20 000000010000000000000092 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 archive_status 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:total 0 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 16 10:09 . 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 55 Mar 16 10:24 .. 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:total 144 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 16 10:09 . 2024-03-16T10:25:00.108 INFO:teuthology.orchestra.run.smithi039.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 16 10:24 .. 2024-03-16T10:25:00.109 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 16 10:24 0000 2024-03-16T10:25:00.110 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-16T10:25:00.120 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-16T10:25:00.120 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T10:25:00.121 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:25:00.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-16T10:25:00.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-16T10:25:00.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-16T10:25:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.774+0000 7f7976600640 1 -- 172.21.15.39:0/1182478351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701039c0 msgr2=0x7f7970103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.774+0000 7f7976600640 1 --2- 172.21.15.39:0/1182478351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701039c0 0x7f7970103e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7958009860 tx=0x7f795802bb70 comp rx=0 tx=0).stop 2024-03-16T10:25:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 -- 172.21.15.39:0/1182478351 shutdown_connections 2024-03-16T10:25:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 --2- 172.21.15.39:0/1182478351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f7970110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 --2- 172.21.15.39:0/1182478351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701039c0 0x7f7970103e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 --2- 172.21.15.39:0/1182478351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701099d0 0x7f7970109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 -- 172.21.15.39:0/1182478351 >> 172.21.15.39:0/1182478351 conn(0x7f79700ff490 msgr2=0x7f79701018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 -- 172.21.15.39:0/1182478351 shutdown_connections 2024-03-16T10:25:00.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.775+0000 7f7976600640 1 -- 172.21.15.39:0/1182478351 wait complete. 2024-03-16T10:25:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 Processor -- start 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 -- start start 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701039c0 0x7f797019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 0x7f797019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7970113db0 con 0x7f79701039c0 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7970113c30 con 0x7f79701099d0 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f7976600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7970113f30 con 0x7f7970104380 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.776+0000 7f796ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 0x7f797019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:38606/0 (socket says 172.21.15.39:38606) 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 -- 172.21.15.39:0/132986701 learned_addr learned my addr 172.21.15.39:0/132986701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796f400640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701039c0 0x7f797019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 -- 172.21.15.39:0/132986701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 msgr2=0x7f797019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 0x7f797019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 -- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701039c0 msgr2=0x7f797019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701039c0 0x7f797019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796ea00640 1 -- 172.21.15.39:0/132986701 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797019f1e0 con 0x7f7970104380 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.777+0000 7f796fe00640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 0x7f797019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f796ea00640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7958037040 tx=0x7f79580045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958041020 con 0x7f7970104380 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f795802c9f0 con 0x7f7970104380 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797019f470 con 0x7f7970104380 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958039040 con 0x7f7970104380 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.778+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79701a6d60 con 0x7f7970104380 2024-03-16T10:25:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.780+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797004f540 con 0x7f7970104380 2024-03-16T10:25:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.781+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f7958044020 con 0x7f7970104380 2024-03-16T10:25:00.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.782+0000 7f7967e00640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 0x7f79540a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:00.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.783+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f79580f08d0 con 0x7f7970104380 2024-03-16T10:25:00.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.783+0000 7f796f400640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 0x7f79540a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.784+0000 7f796f400640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 0x7f79540a4f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7960009b50 tx=0x7f7960005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:00.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:00.788+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f795804b020 con 0x7f7970104380 2024-03-16T10:25:01.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.018+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7970061c20 con 0x7f79540a2a90 2024-03-16T10:25:01.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.029+0000 7f7967e00640 1 -- 172.21.15.39:0/132986701 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7970061c20 con 0x7f79540a2a90 2024-03-16T10:25:01.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-16T00:00:00", "created": "2024-03-16T10:09:14", "first": "2024-03-16T10:10:00", "last": "2024-03-16T10:25:00", "last_pruned": "2024-03-16T10:25:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 msgr2=0x7f79540a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 0x7f79540a4f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7960009b50 tx=0x7f7960005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 msgr2=0x7f797019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7958037040 tx=0x7f79580045b0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 shutdown_connections 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79701099d0 0x7f797019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f79540a2a90 0x7f79540a4f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7970104380 0x7f797019a730 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 --2- 172.21.15.39:0/132986701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79701039c0 0x7f797019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.032+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 >> 172.21.15.39:0/132986701 conn(0x7f79700ff490 msgr2=0x7f79700ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.033+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 shutdown_connections 2024-03-16T10:25:01.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.033+0000 7f7976600640 1 -- 172.21.15.39:0/132986701 wait complete. 2024-03-16T10:25:01.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 -- 172.21.15.39:0/1516163501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 msgr2=0x7f1a6010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 --2- 172.21.15.39:0/1516163501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6010db10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009900 tx=0x7f1a4c02bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 -- 172.21.15.39:0/1516163501 shutdown_connections 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 --2- 172.21.15.39:0/1516163501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a6010e050 0x7f1a60110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 --2- 172.21.15.39:0/1516163501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6010db10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 --2- 172.21.15.39:0/1516163501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a60073b90 0x7f1a60073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 -- 172.21.15.39:0/1516163501 >> 172.21.15.39:0/1516163501 conn(0x7f1a600fd2a0 msgr2=0x7f1a600ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:01.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.302+0000 7f1a67800640 1 -- 172.21.15.39:0/1516163501 shutdown_connections 2024-03-16T10:25:01.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.303+0000 7f1a67800640 1 -- 172.21.15.39:0/1516163501 wait complete. 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.303+0000 7f1a67800640 1 Processor -- start 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 -- start start 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 0x7f1a6019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60113d70 con 0x7f1a6010e050 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60113bf0 con 0x7f1a60073b90 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a67800640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60113ef0 con 0x7f1a600744b0 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a64e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a65800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 0x7f1a6019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a64e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59738/0 (socket says 172.21.15.39:59738) 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a5fe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.304+0000 7f1a64e00640 1 -- 172.21.15.39:0/2351451809 learned_addr learned my addr 172.21.15.39:0/2351451809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 -- 172.21.15.39:0/2351451809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 msgr2=0x7f1a6019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 -- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 msgr2=0x7f1a6019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 0x7f1a6019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 -- 172.21.15.39:0/2351451809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6019f230 con 0x7f1a60073b90 2024-03-16T10:25:01.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a65800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 0x7f1a6019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:01.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a64e00640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400e9f0 tx=0x7f1a5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.305+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400cd80 con 0x7f1a60073b90 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.306+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a54010910 con 0x7f1a60073b90 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.306+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6019f520 con 0x7f1a60073b90 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.306+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54018990 con 0x7f1a60073b90 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.306+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a601a6db0 con 0x7f1a60073b90 2024-03-16T10:25:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.307+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6004f540 con 0x7f1a60073b90 2024-03-16T10:25:01.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.309+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f1a54010ac0 con 0x7f1a60073b90 2024-03-16T10:25:01.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.311+0000 7f1a5d600640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 0x7f1a480a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.311+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f1a540cf690 con 0x7f1a60073b90 2024-03-16T10:25:01.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.311+0000 7f1a5fe00640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 0x7f1a480a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.312+0000 7f1a5fe00640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 0x7f1a480a4f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c02efd0 tx=0x7f1a4c02cf90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:01.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.317+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1a5409b990 con 0x7f1a60073b90 2024-03-16T10:25:01.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.573+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 --> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1a60061c20 con 0x7f1a480a2a90 2024-03-16T10:25:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.586+0000 7f1a5d600640 1 -- 172.21.15.39:0/2351451809 <== mgr.25460 v2:172.21.15.39:6800/2004847378 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1a60061c20 con 0x7f1a480a2a90 2024-03-16T10:25:01.586 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 msgr2=0x7f1a480a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 0x7f1a480a4f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c02efd0 tx=0x7f1a4c02cf90 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 msgr2=0x7f1a6019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400e9f0 tx=0x7f1a5400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 shutdown_connections 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a6010e050 0x7f1a6019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f1a480a2a90 0x7f1a480a4f50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a600744b0 0x7f1a6019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 --2- 172.21.15.39:0/2351451809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a60073b90 0x7f1a6019a290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.588+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 >> 172.21.15.39:0/2351451809 conn(0x7f1a600fd2a0 msgr2=0x7f1a600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.589+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 shutdown_connections 2024-03-16T10:25:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.589+0000 7f1a67800640 1 -- 172.21.15.39:0/2351451809 wait complete. 2024-03-16T10:25:01.609 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-16T10:25:01.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:25:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:01 smithi039 ceph-mon[31510]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-16T10:25:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:01 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:25:01.757 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:01 smithi194 ceph-mon[36445]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-16T10:25:01.758 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:01 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 -- 172.21.15.39:0/1974001832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42341027d0 msgr2=0x7f4234102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 --2- 172.21.15.39:0/1974001832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42341027d0 0x7f4234102bb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4224009900 tx=0x7f422402bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 -- 172.21.15.39:0/1974001832 shutdown_connections 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 --2- 172.21.15.39:0/1974001832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f423410bba0 0x7f423410dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 --2- 172.21.15.39:0/1974001832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f423410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 --2- 172.21.15.39:0/1974001832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42341027d0 0x7f4234102bb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 -- 172.21.15.39:0/1974001832 >> 172.21.15.39:0/1974001832 conn(0x7f42340fbf70 msgr2=0x7f42340fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 -- 172.21.15.39:0/1974001832 shutdown_connections 2024-03-16T10:25:01.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.834+0000 7f4233200640 1 -- 172.21.15.39:0/1974001832 wait complete. 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.835+0000 7f4233200640 1 Processor -- start 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.835+0000 7f4233200640 1 -- start start 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f4234103b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 0x7f42341040c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4234111960 con 0x7f42341030f0 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42341117e0 con 0x7f423410bba0 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4233200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4234111ae0 con 0x7f42341027d0 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4231e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4231e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:38644/0 (socket says 172.21.15.39:38644) 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4232800640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 0x7f42341040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.836+0000 7f4231e00640 1 -- 172.21.15.39:0/32491195 learned_addr learned my addr 172.21.15.39:0/32491195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231400640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f4234103b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 -- 172.21.15.39:0/32491195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 msgr2=0x7f42341040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 0x7f42341040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 -- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 msgr2=0x7f4234103b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f4234103b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 -- 172.21.15.39:0/32491195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42341aa6c0 con 0x7f42341027d0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4232800640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 0x7f42341040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231400640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f4234103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4231e00640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224042020 con 0x7f42341027d0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42341aa950 con 0x7f42341027d0 2024-03-16T10:25:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42240054e0 con 0x7f42341027d0 2024-03-16T10:25:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.837+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224003b80 con 0x7f42341027d0 2024-03-16T10:25:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.838+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42341aaea0 con 0x7f42341027d0 2024-03-16T10:25:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.839+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4234069180 con 0x7f42341027d0 2024-03-16T10:25:01.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.841+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f4224045020 con 0x7f42341027d0 2024-03-16T10:25:01.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.842+0000 7f422aa00640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 0x7f42180a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:01.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.842+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f42240f0e00 con 0x7f42341027d0 2024-03-16T10:25:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.843+0000 7f4231400640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 0x7f42180a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.843+0000 7f4231400640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 0x7f42180a4e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4234104e00 tx=0x7f4220005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:01.846+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f422404c020 con 0x7f42341027d0 2024-03-16T10:25:01.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:01 smithi105 ceph-mon[36559]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 4.3 MiB/s wr, 500 op/s 2024-03-16T10:25:01.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:01 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:25:02.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.108+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42341ab150 con 0x7f42341027d0 2024-03-16T10:25:02.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.109+0000 7f422aa00640 1 -- 172.21.15.39:0/32491195 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25415 (secure 0 0 0) 0x7f42240bd800 con 0x7f42341027d0 2024-03-16T10:25:02.110 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:02.111 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":115,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:24:12.269037+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3265323877":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/2375807750":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/332905917":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/2792410890":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:25:02.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 msgr2=0x7f42180a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 0x7f42180a4e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4234104e00 tx=0x7f4220005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 msgr2=0x7f4234103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224004360 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 shutdown_connections 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f423410bba0 0x7f42341040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42341030f0 0x7f4234103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f42180a2970 0x7f42180a4e30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 --2- 172.21.15.39:0/32491195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42341027d0 0x7f4234103640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 >> 172.21.15.39:0/32491195 conn(0x7f42340fbf70 msgr2=0x7f423410dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 shutdown_connections 2024-03-16T10:25:02.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.113+0000 7f4233200640 1 -- 172.21.15.39:0/32491195 wait complete. 2024-03-16T10:25:02.142 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:25:02.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 -- 172.21.15.39:0/2183753421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 msgr2=0x7f16a0106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 --2- 172.21.15.39:0/2183753421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0106a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1694009930 tx=0x7f169402bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 -- 172.21.15.39:0/2183753421 shutdown_connections 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 --2- 172.21.15.39:0/2183753421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 0x7f16a010d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 --2- 172.21.15.39:0/2183753421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 --2- 172.21.15.39:0/2183753421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0106a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 -- 172.21.15.39:0/2183753421 >> 172.21.15.39:0/2183753421 conn(0x7f16a00fc030 msgr2=0x7f16a00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 -- 172.21.15.39:0/2183753421 shutdown_connections 2024-03-16T10:25:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.380+0000 7f16a7000640 1 -- 172.21.15.39:0/2183753421 wait complete. 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.381+0000 7f16a7000640 1 Processor -- start 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 -- start start 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 0x7f16a0102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0110930 con 0x7f16a0100f20 2024-03-16T10:25:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f16a01107b0 con 0x7f16a0100560 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a7000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0110ab0 con 0x7f16a0106620 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 0x7f16a0102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a6600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a5c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59786/0 (socket says 172.21.15.39:59786) 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.382+0000 7f16a5c00640 1 -- 172.21.15.39:0/1135120038 learned_addr learned my addr 172.21.15.39:0/1135120038 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 -- 172.21.15.39:0/1135120038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 msgr2=0x7f16a0102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 -- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 msgr2=0x7f16a0102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 0x7f16a0102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 -- 172.21.15.39:0/1135120038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a01a6360 con 0x7f16a0100560 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a6600640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f16a5c00640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f1694004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:02.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694043020 con 0x7f16a0100560 2024-03-16T10:25:02.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.383+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16940054e0 con 0x7f16a0100560 2024-03-16T10:25:02.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.384+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a01a65f0 con 0x7f16a0100560 2024-03-16T10:25:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.384+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694003b80 con 0x7f16a0100560 2024-03-16T10:25:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.384+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a6b90 con 0x7f16a0100560 2024-03-16T10:25:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.385+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a00684f0 con 0x7f16a0100560 2024-03-16T10:25:02.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.387+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f1694046020 con 0x7f16a0100560 2024-03-16T10:25:02.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.388+0000 7f169e000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 0x7f16880a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.388+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f16940f1190 con 0x7f16a0100560 2024-03-16T10:25:02.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.388+0000 7f16a5200640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 0x7f16880a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.389+0000 7f16a5200640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 0x7f16880a4ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16a0103c10 tx=0x7f168c005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.392+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f169404d020 con 0x7f16a0100560 2024-03-16T10:25:02.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:02 smithi039 ceph-mon[31510]: from='client.25549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:02 smithi039 ceph-mon[31510]: from='client.25556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:02 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/32491195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:02.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.675+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16a01a6d60 con 0x7f16a0100560 2024-03-16T10:25:02.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.676+0000 7f169e000640 1 -- 172.21.15.39:0/1135120038 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f16940bdb90 con 0x7f16a0100560 2024-03-16T10:25:02.676 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:25:02.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 msgr2=0x7f16880a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 0x7f16880a4ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16a0103c10 tx=0x7f168c005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 msgr2=0x7f16a01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f1694004360 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 shutdown_connections 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f16880a2a20 0x7f16880a4ee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f16a0106620 0x7f16a0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16a0100f20 0x7f16a0102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 --2- 172.21.15.39:0/1135120038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16a0100560 0x7f16a01021d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 >> 172.21.15.39:0/1135120038 conn(0x7f16a00fc030 msgr2=0x7f16a00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 shutdown_connections 2024-03-16T10:25:02.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.679+0000 7f16a7000640 1 -- 172.21.15.39:0/1135120038 wait complete. 2024-03-16T10:25:02.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:25:02.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:02 smithi105 ceph-mon[36559]: from='client.25549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:02 smithi105 ceph-mon[36559]: from='client.25556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:02 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/32491195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 -- 172.21.15.39:0/813017568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024109870 msgr2=0x7f1024109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 --2- 172.21.15.39:0/813017568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024109870 0x7f1024109c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1014009900 tx=0x7f101402bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 -- 172.21.15.39:0/813017568 shutdown_connections 2024-03-16T10:25:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 --2- 172.21.15.39:0/813017568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024104220 0x7f1024110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 --2- 172.21.15.39:0/813017568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024103860 0x7f1024103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 --2- 172.21.15.39:0/813017568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024109870 0x7f1024109c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 -- 172.21.15.39:0/813017568 >> 172.21.15.39:0/813017568 conn(0x7f10240ff2f0 msgr2=0x7f1024101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 -- 172.21.15.39:0/813017568 shutdown_connections 2024-03-16T10:25:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.919+0000 7f1029000640 1 -- 172.21.15.39:0/813017568 wait complete. 2024-03-16T10:25:02.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 Processor -- start 2024-03-16T10:25:02.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 -- start start 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 0x7f102419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024109870 0x7f102419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1024113c90 con 0x7f1024109870 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1024113b10 con 0x7f1024104220 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.920+0000 7f1029000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1024113e10 con 0x7f1024103860 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1023400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 0x7f102419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59800/0 (socket says 172.21.15.39:59800) 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 -- 172.21.15.39:0/1852868473 learned_addr learned my addr 172.21.15.39:0/1852868473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1023e00640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024109870 0x7f102419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 -- 172.21.15.39:0/1852868473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 msgr2=0x7f102419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 0x7f102419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 -- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024109870 msgr2=0x7f102419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:02.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024109870 0x7f102419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:02.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1022a00640 1 -- 172.21.15.39:0/1852868473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102419f180 con 0x7f1024104220 2024-03-16T10:25:02.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.921+0000 7f1023400640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 0x7f102419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:02.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f1022a00640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9f0 tx=0x7f101000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101000ccc0 con 0x7f1024104220 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1010010910 con 0x7f1024104220 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102419f470 con 0x7f1024104220 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10100188a0 con 0x7f1024104220 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.922+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10241a6d00 con 0x7f1024104220 2024-03-16T10:25:02.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.923+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10240684f0 con 0x7f1024104220 2024-03-16T10:25:02.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.926+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f1010010ac0 con 0x7f1024104220 2024-03-16T10:25:02.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.928+0000 7f101be00640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 0x7f10080a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.928+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f10100cf650 con 0x7f1024104220 2024-03-16T10:25:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.929+0000 7f1023400640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 0x7f10080a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.929+0000 7f1023400640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 0x7f10080a4eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f101402efd0 tx=0x7f1014000bc0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:02.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:02.931+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f101009bf40 con 0x7f1024104220 2024-03-16T10:25:02.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:02 smithi194 ceph-mon[36445]: from='client.25549 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:02 smithi194 ceph-mon[36445]: from='client.25556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:25:02.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:02 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/32491195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:03.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.150+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f102410bdd0 con 0x7f1024104220 2024-03-16T10:25:03.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.150+0000 7f101be00640 1 -- 172.21.15.39:0/1852868473 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f101009b840 con 0x7f1024104220 2024-03-16T10:25:03.151 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:25:03.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 msgr2=0x7f10080a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 0x7f10080a4eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f101402efd0 tx=0x7f1014000bc0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 msgr2=0x7f102419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f101000e9f0 tx=0x7f101000edf0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 shutdown_connections 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024109870 0x7f102419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1024104220 0x7f102419a6d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f10080a29f0 0x7f10080a4eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 --2- 172.21.15.39:0/1852868473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1024103860 0x7f102419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 >> 172.21.15.39:0/1852868473 conn(0x7f10240ff2f0 msgr2=0x7f10240ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 shutdown_connections 2024-03-16T10:25:03.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.154+0000 7f1029000640 1 -- 172.21.15.39:0/1852868473 wait complete. 2024-03-16T10:25:03.175 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.391+0000 7f5b93e00640 1 -- 172.21.15.39:0/3711701297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94102810 msgr2=0x7f5b94102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.391+0000 7f5b93e00640 1 --2- 172.21.15.39:0/3711701297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94102810 0x7f5b94102bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009900 tx=0x7f5b8402bcc0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 -- 172.21.15.39:0/3711701297 shutdown_connections 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 --2- 172.21.15.39:0/3711701297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b9410bc30 0x7f5b9410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 --2- 172.21.15.39:0/3711701297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b94103130 0x7f5b9410b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 --2- 172.21.15.39:0/3711701297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94102810 0x7f5b94102bf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 -- 172.21.15.39:0/3711701297 >> 172.21.15.39:0/3711701297 conn(0x7f5b940fc030 msgr2=0x7f5b940fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 -- 172.21.15.39:0/3711701297 shutdown_connections 2024-03-16T10:25:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.392+0000 7f5b93e00640 1 -- 172.21.15.39:0/3711701297 wait complete. 2024-03-16T10:25:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.393+0000 7f5b93e00640 1 Processor -- start 2024-03-16T10:25:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.393+0000 7f5b93e00640 1 -- start start 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 0x7f5b94198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9410bc30 0x7f5b9419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94111a80 con 0x7f5b9410bc30 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94111900 con 0x7f5b94102810 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94111c00 con 0x7f5b94103130 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b92a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b92000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 0x7f5b94198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b92a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59824/0 (socket says 172.21.15.39:59824) 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b92a00640 1 -- 172.21.15.39:0/2923907992 learned_addr learned my addr 172.21.15.39:0/2923907992 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.394+0000 7f5b93400640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9410bc30 0x7f5b9419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 -- 172.21.15.39:0/2923907992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 msgr2=0x7f5b94198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 0x7f5b94198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 -- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9410bc30 msgr2=0x7f5b9419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9410bc30 0x7f5b9419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 -- 172.21.15.39:0/2923907992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9419cf10 con 0x7f5b94102810 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92000640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 0x7f5b94198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:03.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b92a00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84042020 con 0x7f5b94102810 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84005000 con 0x7f5b94102810 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9419d1a0 con 0x7f5b94102810 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.395+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84003b20 con 0x7f5b94102810 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.396+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b94100390 con 0x7f5b94102810 2024-03-16T10:25:03.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.397+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b940684f0 con 0x7f5b94102810 2024-03-16T10:25:03.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.398+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f5b84045020 con 0x7f5b94102810 2024-03-16T10:25:03.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.400+0000 7f5b8b400640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 0x7f5b780a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.400+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f5b840f0d80 con 0x7f5b94102810 2024-03-16T10:25:03.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.400+0000 7f5b92000640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 0x7f5b780a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.401+0000 7f5b92000640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 0x7f5b780a4e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b800095f0 tx=0x7f5b80012040 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:03.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.404+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5b8404c020 con 0x7f5b94102810 2024-03-16T10:25:03.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.639+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5b94106cd0 con 0x7f5b94102810 2024-03-16T10:25:03.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.640+0000 7f5b8b400640 1 -- 172.21.15.39:0/2923907992 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7f5b840bd780 con 0x7f5b94102810 2024-03-16T10:25:03.640 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:25:03.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 msgr2=0x7f5b780a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 0x7f5b780a4e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b800095f0 tx=0x7f5b80012040 comp rx=0 tx=0).stop 2024-03-16T10:25:03.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 msgr2=0x7f5b94197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b84004360 comp rx=0 tx=0).stop 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 shutdown_connections 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9410bc30 0x7f5b9419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f5b780a2970 0x7f5b780a4e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b94103130 0x7f5b94198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 --2- 172.21.15.39:0/2923907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b94102810 0x7f5b94197f20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 >> 172.21.15.39:0/2923907992 conn(0x7f5b940fc030 msgr2=0x7f5b940fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 shutdown_connections 2024-03-16T10:25:03.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.643+0000 7f5b93e00640 1 -- 172.21.15.39:0/2923907992 wait complete. 2024-03-16T10:25:03.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:25:03.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 508 op/s 2024-03-16T10:25:03.743 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:25:03.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:25:03.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:25:03.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1135120038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:03.744 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:03 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1852868473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 508 op/s 2024-03-16T10:25:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:25:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:25:03.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:25:03.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1135120038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:03.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:03 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1852868473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 -- 172.21.15.39:0/2632217395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c0744b0 msgr2=0x7ff24c10daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 --2- 172.21.15.39:0/2632217395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c0744b0 0x7ff24c10daf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff234009900 tx=0x7ff23402bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 -- 172.21.15.39:0/2632217395 shutdown_connections 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 --2- 172.21.15.39:0/2632217395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 0x7ff24c110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 --2- 172.21.15.39:0/2632217395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c0744b0 0x7ff24c10daf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 --2- 172.21.15.39:0/2632217395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c073b90 0x7ff24c073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 -- 172.21.15.39:0/2632217395 >> 172.21.15.39:0/2632217395 conn(0x7ff24c078be0 msgr2=0x7ff24c078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 -- 172.21.15.39:0/2632217395 shutdown_connections 2024-03-16T10:25:03.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.886+0000 7ff252200640 1 -- 172.21.15.39:0/2632217395 wait complete. 2024-03-16T10:25:03.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.887+0000 7ff252200640 1 Processor -- start 2024-03-16T10:25:03.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.887+0000 7ff252200640 1 -- start start 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 0x7ff24c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 0x7ff24c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c113d20 con 0x7ff24c0744b0 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c113ba0 con 0x7ff24c10e030 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff252200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c113ea0 con 0x7ff24c073b90 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24be00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 0x7ff24c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:38724/0 (socket says 172.21.15.39:38724) 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 -- 172.21.15.39:0/1126757826 learned_addr learned my addr 172.21.15.39:0/1126757826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff243e00640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 0x7ff24c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 -- 172.21.15.39:0/1126757826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 msgr2=0x7ff24c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 0x7ff24c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 -- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 msgr2=0x7ff24c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 0x7ff24c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff24b400640 1 -- 172.21.15.39:0/1126757826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19f210 con 0x7ff24c073b90 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.888+0000 7ff243e00640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 0x7ff24c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff24b400640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00e9f0 tx=0x7ff23c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00cd80 con 0x7ff24c073b90 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23c010910 con 0x7ff24c073b90 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19f500 con 0x7ff24c073b90 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c018990 con 0x7ff24c073b90 2024-03-16T10:25:03.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.889+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a6d90 con 0x7ff24c073b90 2024-03-16T10:25:03.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.890+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c068a20 con 0x7ff24c073b90 2024-03-16T10:25:03.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.892+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7ff23c010ac0 con 0x7ff24c073b90 2024-03-16T10:25:03.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.893+0000 7ff248c00640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 0x7ff2300a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.893+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7ff23c0cf660 con 0x7ff24c073b90 2024-03-16T10:25:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.894+0000 7ff243e00640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 0x7ff2300a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:03.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.894+0000 7ff243e00640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 0x7ff2300a4e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff24c100970 tx=0x7ff234005a20 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:03.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:03.897+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff23c09bfb0 con 0x7ff24c073b90 2024-03-16T10:25:03.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 508 op/s 2024-03-16T10:25:03.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:25:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:25:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: from='mgr.25460 ' entity='mgr.x' 2024-03-16T10:25:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1135120038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:03.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:03 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1852868473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.102+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff24c02d1b0 con 0x7ff24c073b90 2024-03-16T10:25:04.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.103+0000 7ff248c00640 1 -- 172.21.15.39:0/1126757826 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) ==== 103+0+11 (secure 0 0 0) 0x7ff23c09b8b0 con 0x7ff24c073b90 2024-03-16T10:25:04.103 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:25:04.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 msgr2=0x7ff2300a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:04.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 0x7ff2300a4e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff24c100970 tx=0x7ff234005a20 comp rx=0 tx=0).stop 2024-03-16T10:25:04.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 msgr2=0x7ff24c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00e9f0 tx=0x7ff23c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 shutdown_connections 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff24c10e030 0x7ff24c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff24c0744b0 0x7ff24c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7ff2300a2970 0x7ff2300a4e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 --2- 172.21.15.39:0/1126757826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff24c073b90 0x7ff24c19a220 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 >> 172.21.15.39:0/1126757826 conn(0x7ff24c078be0 msgr2=0x7ff24c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 shutdown_connections 2024-03-16T10:25:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.106+0000 7ff252200640 1 -- 172.21.15.39:0/1126757826 wait complete. 2024-03-16T10:25:04.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-16T10:25:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 -- 172.21.15.39:0/1736639477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c60073b80 msgr2=0x7f8c60073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 --2- 172.21.15.39:0/1736639477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c60073b80 0x7f8c60073f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009900 tx=0x7f8c5002bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 -- 172.21.15.39:0/1736639477 shutdown_connections 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 --2- 172.21.15.39:0/1736639477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 0x7f8c601149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 --2- 172.21.15.39:0/1736639477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c600744a0 0x7f8c601120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 --2- 172.21.15.39:0/1736639477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c60073b80 0x7f8c60073f60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 -- 172.21.15.39:0/1736639477 >> 172.21.15.39:0/1736639477 conn(0x7f8c600fd0c0 msgr2=0x7f8c600ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 -- 172.21.15.39:0/1736639477 shutdown_connections 2024-03-16T10:25:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.341+0000 7f8c64c00640 1 -- 172.21.15.39:0/1736639477 wait complete. 2024-03-16T10:25:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.343+0000 7f8c64c00640 1 Processor -- start 2024-03-16T10:25:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.343+0000 7f8c64c00640 1 -- start start 2024-03-16T10:25:04.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.343+0000 7f8c64c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:04.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.343+0000 7f8c64c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 0x7f8c6019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:04.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c64c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 0x7f8c601a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:04.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c64c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60118030 con 0x7f8c601125e0 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c64c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60117eb0 con 0x7f8c60073b80 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c64c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601181b0 con 0x7f8c600744a0 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5ea00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 0x7f8c6019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:59858/0 (socket says 172.21.15.39:59858) 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 -- 172.21.15.39:0/2766781496 learned_addr learned my addr 172.21.15.39:0/2766781496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5fe00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 0x7f8c601a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 -- 172.21.15.39:0/2766781496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 msgr2=0x7f8c6019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 0x7f8c6019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 -- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 msgr2=0x7f8c601a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 0x7f8c601a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5f400640 1 -- 172.21.15.39:0/2766781496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c601a3500 con 0x7f8c60073b80 2024-03-16T10:25:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.344+0000 7f8c5ea00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 0x7f8c6019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c5f400640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c50000c00 tx=0x7f8c50004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50042020 con 0x7f8c60073b80 2024-03-16T10:25:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c500054e0 con 0x7f8c60073b80 2024-03-16T10:25:04.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c601a3790 con 0x7f8c60073b80 2024-03-16T10:25:04.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50003b50 con 0x7f8c60073b80 2024-03-16T10:25:04.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.345+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601ab080 con 0x7f8c60073b80 2024-03-16T10:25:04.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.347+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c600684f0 con 0x7f8c60073b80 2024-03-16T10:25:04.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.348+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f8c50045020 con 0x7f8c60073b80 2024-03-16T10:25:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.350+0000 7f8c57e00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 0x7f8c440a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.350+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9562+0+0 (secure 0 0 0) 0x7f8c500f0d10 con 0x7f8c60073b80 2024-03-16T10:25:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.350+0000 7f8c5ea00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 0x7f8c440a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.350+0000 7f8c5ea00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 0x7f8c440a4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c60105390 tx=0x7f8c4c005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:04.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.353+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8c500f6020 con 0x7f8c60073b80 2024-03-16T10:25:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2923907992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:04.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:04 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1126757826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:04.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.764+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8c60104a30 con 0x7f8c60073b80 2024-03-16T10:25:04.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.779+0000 7f8c57e00640 1 -- 172.21.15.39:0/2766781496 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+740362 (secure 0 0 0) 0x7f8c5004c020 con 0x7f8c60073b80 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 27, 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 25460, 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-03-16T10:25:04.784 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2004847378 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2004847378 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/2004847378", 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2024-03-16T10:09:13.857097+0000", 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540701547738038271, 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2024-03-16T10:25:04.785 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 25457, 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:04.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-03-16T10:25:04.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-03-16T10:25:04.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T10:25:04.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T10:25:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T10:25:04.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:04.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.809 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.812 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-03-16T10:25:04.815 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:04.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-03-16T10:25:04.820 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.822 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.824 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-03-16T10:25:04.827 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-03-16T10:25:04.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-03-16T10:25:04.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.836 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-03-16T10:25:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-03-16T10:25:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-03-16T10:25:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T10:25:04.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-03-16T10:25:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-03-16T10:25:04.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.853 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T10:25:04.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.856 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T10:25:04.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T10:25:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-03-16T10:25:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-03-16T10:25:04.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-03-16T10:25:04.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-03-16T10:25:04.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.878 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T10:25:04.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.884 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-03-16T10:25:04.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.905 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.906 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.912 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.913 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.915 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.917 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T10:25:04.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T10:25:04.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T10:25:04.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T10:25:04.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.931 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.932 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2923907992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:04.935 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:04 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1126757826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-03-16T10:25:04.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:04.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.953 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.954 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-03-16T10:25:04.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.960 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.968 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.969 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-03-16T10:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.972 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-03-16T10:25:04.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:04.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T10:25:04.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.984 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.986 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:04.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:04.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:04.994 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.995 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:04.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.003 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.005 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.008 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.012 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.017 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-03-16T10:25:05.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-03-16T10:25:05.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-03-16T10:25:05.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.022 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-03-16T10:25:05.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.025 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.029 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.035 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.036 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.040 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.042 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.045 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-03-16T10:25:05.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.051 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.053 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.054 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.057 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.059 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.060 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.063 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.065 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.073 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.077 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T10:25:05.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.080 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.083 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.091 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-03-16T10:25:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.095 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:05.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.108 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-03-16T10:25:05.112 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.113 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.119 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.122 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-03-16T10:25:05.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.128 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.134 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.142 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.145 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.151 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.165 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.170 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:05.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.180 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.181 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.185 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.188 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.189 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T10:25:05.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.194 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.196 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.197 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.203 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-03-16T10:25:05.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-03-16T10:25:05.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-03-16T10:25:05.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-03-16T10:25:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.209 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.213 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.217 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.219 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T10:25:05.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.221 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 25462, 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.226 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.232 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.234 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.235 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:05.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-03-16T10:25:05.239 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T10:25:05.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.244 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T10:25:05.249 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.250 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.253 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-03-16T10:25:05.255 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-03-16T10:25:05.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-03-16T10:25:05.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-03-16T10:25:05.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-03-16T10:25:05.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.262 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-03-16T10:25:05.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-03-16T10:25:05.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-03-16T10:25:05.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.267 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T10:25:05.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T10:25:05.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.273 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T10:25:05.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.285 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.286 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-03-16T10:25:05.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-03-16T10:25:05.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T10:25:05.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.293 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T10:25:05.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T10:25:05.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T10:25:05.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.306 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T10:25:05.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T10:25:05.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.313 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.316 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.319 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.323 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.336 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T10:25:05.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.342 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.353 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T10:25:05.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T10:25:05.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.362 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.366 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.805+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 msgr2=0x7f8c440a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.805+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 0x7f8c440a4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c60105390 tx=0x7f8c4c005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.805+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 msgr2=0x7f8c6019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.805+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c50000c00 tx=0x7f8c50004360 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.805+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 shutdown_connections 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c601125e0 0x7f8c601a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8c440a29f0 0x7f8c440a4eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c600744a0 0x7f8c6019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 --2- 172.21.15.39:0/2766781496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c60073b80 0x7f8c6019e510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 >> 172.21.15.39:0/2766781496 conn(0x7f8c600fd0c0 msgr2=0x7f8c600fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 shutdown_connections 2024-03-16T10:25:05.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:04.806+0000 7f8c64c00640 1 -- 172.21.15.39:0/2766781496 wait complete. 2024-03-16T10:25:05.373 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2923907992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:05.373 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:04 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1126757826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.376 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.381 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:05.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-03-16T10:25:05.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:05.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.391 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.392 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.400 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.406 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.419 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:05.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.427 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.428 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.429 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.433 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.434 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.439 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.441 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-03-16T10:25:05.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.443 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.446 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.450 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.458 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.459 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.461 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.462 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.463 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.464 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.466 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.471 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.474 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.475 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.477 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-03-16T10:25:05.482 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.483 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.484 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.487 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.491 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.492 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.495 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.497 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.498 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.500 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:05.501 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-03-16T10:25:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.505 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-03-16T10:25:05.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-03-16T10:25:05.511 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.513 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.514 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.516 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.523 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:05.525 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-03-16T10:25:05.526 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.528 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.529 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.530 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.533 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.534 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.543 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.544 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.546 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.547 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.549 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.550 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.554 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.555 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.557 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.558 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-03-16T10:25:05.559 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.562 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.563 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.571 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-03-16T10:25:05.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.574 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.581 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.582 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.585 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.586 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.587 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.588 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.591 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.593 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.595 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.597 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.598 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.600 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-03-16T10:25:05.602 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.603 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-03-16T10:25:05.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:05.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-03-16T10:25:05.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.609 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.610 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.612 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.613 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.615 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.617 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.618 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.619 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.621 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.622 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.624 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.625 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.626 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.627 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.630 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T10:25:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.632 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-03-16T10:25:05.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-03-16T10:25:05.634 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.635 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.637 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.639 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.640 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.642 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.643 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "restful", 2024-03-16T10:25:05.645 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_schedule" 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.647 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.648 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.649 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.650 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.652 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.654 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T10:25:05.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.659 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.662 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.663 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-03-16T10:25:05.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.665 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-03-16T10:25:05.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.667 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.668 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.670 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.671 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.674 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.678 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.679 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.680 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.681 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.683 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.685 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.686 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.687 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.690 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.691 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.693 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-03-16T10:25:05.695 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.696 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.697 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.699 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-03-16T10:25:05.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.701 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.702 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.703 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.704 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.706 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.708 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-03-16T10:25:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.712 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.713 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.714 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.717 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.718 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.719 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.720 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.721 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.722 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.724 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T10:25:05.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.726 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.727 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-03-16T10:25:05.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-03-16T10:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.732 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-03-16T10:25:05.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.735 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-03-16T10:25:05.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-03-16T10:25:05.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-03-16T10:25:05.738 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-03-16T10:25:05.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.744 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.746 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.748 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.749 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.750 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.757 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.758 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.759 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.760 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.763 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.764 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.766 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.771 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.774 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T10:25:05.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T10:25:05.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T10:25:05.778 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T10:25:05.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.780 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.781 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.785 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.786 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.787 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T10:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T10:25:05.790 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.793 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:05.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.795 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.799 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.802 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.805 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.806 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.807 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.809 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:05.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.814 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-03-16T10:25:05.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.819 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.820 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-03-16T10:25:05.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.828 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.829 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.831 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-03-16T10:25:05.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.833 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.834 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.835 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-03-16T10:25:05.836 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-03-16T10:25:05.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.843 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-03-16T10:25:05.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-03-16T10:25:05.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-03-16T10:25:05.847 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-03-16T10:25:05.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-03-16T10:25:05.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.850 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.851 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.855 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.859 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.860 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.863 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.864 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.869 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.870 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.871 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.882 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.885 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.886 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.890 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.891 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.892 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.893 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.896 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.897 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.903 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-03-16T10:25:05.904 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.908 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.909 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.910 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.911 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.913 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.915 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.916 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.918 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.920 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.922 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.924 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.925 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.928 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-03-16T10:25:05.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.930 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.931 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.936 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.937 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.943 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-03-16T10:25:05.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-03-16T10:25:05.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.946 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.950 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.958 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.960 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.961 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-03-16T10:25:05.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.967 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.968 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.972 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.974 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-03-16T10:25:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.989 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.990 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.991 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-03-16T10:25:05.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.996 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:05.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:05.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:05.999 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.001 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.009 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.012 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.013 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.018 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T10:25:06.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.037 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T10:25:06.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.042 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.043 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.049 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.051 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-03-16T10:25:06.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:06.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-03-16T10:25:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.061 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-03-16T10:25:06.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2024-03-16T10:25:06.069 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-03-16T10:25:06.070 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-03-16T10:25:06.071 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-03-16T10:25:06.072 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "squid": [ 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-03-16T10:25:06.073 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 95, 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2681839994 2024-03-16T10:25:06.074 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 817274836 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-03-16T10:25:06.075 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2912191062 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3469869338 2024-03-16T10:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1799254113 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-03-16T10:25:06.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2227869899 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-03-16T10:25:06.078 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:25:06.084 DEBUG:tasks.check_counter:Getting stats from a 2024-03-16T10:25:06.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-16T10:25:06.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:05 smithi039 ceph-mon[31510]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 3.9 MiB/s wr, 472 op/s 2024-03-16T10:25:06.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:05 smithi039 ceph-mon[31510]: osdmap e116: 12 total, 12 up, 12 in 2024-03-16T10:25:06.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:05 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2766781496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:05 smithi105 ceph-mon[36559]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 3.9 MiB/s wr, 472 op/s 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:05 smithi105 ceph-mon[36559]: osdmap e116: 12 total, 12 up, 12 in 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:05 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2766781496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:05 smithi194 ceph-mon[36445]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 3.9 MiB/s wr, 472 op/s 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:05 smithi194 ceph-mon[36445]: osdmap e116: 12 total, 12 up, 12 in 2024-03-16T10:25:06.086 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:05 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2766781496' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T10:25:06.511 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:25:07.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:07 smithi039 ceph-mon[31510]: pgmap v500: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-16T10:25:07.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:07 smithi105 ceph-mon[36559]: pgmap v500: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-16T10:25:07.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:07 smithi194 ceph-mon[36445]: pgmap v500: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.4 MiB/s wr, 317 op/s 2024-03-16T10:25:09.080 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:25:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 5610, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 6346, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2217119, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 6263112, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 11, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 1.345106756, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.758267374, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.610593160, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.073190139, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6345, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 2.046461786, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000322531 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 9383, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.003742820, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000398 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 452, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 479, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 593956, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 621361, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 22, 2024-03-16T10:25:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.108921584, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.040674035, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.060271713, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.006531544, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 473, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.179231353, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000378924 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 472, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000161031, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000341 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 316720, 2024-03-16T10:25:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 247200 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 738, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 892, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2480606, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 1175110, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 13, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.227246971, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.075255625, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.166716372, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.024307105, 2024-03-16T10:25:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 890, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.330842176, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000371732 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 680, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000418277, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000615 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1977232, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 632816 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-03-16T10:25:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1430, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.900344633, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000629611 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000010293, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000005146 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 617, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 582, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 582, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3.503127845, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.006019120 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-03-16T10:25:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 35, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 5, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 20, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 1, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 337, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 18, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 195, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 124, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 309, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 420, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 65, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 65, 2024-03-16T10:25:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 16, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 2072, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 1505, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 34, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 37, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 2, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 90, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 14737, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 3, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 9, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 345, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 7, 2024-03-16T10:25:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 335, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 1004, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 1125196203, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 5, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 12, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 225, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 459, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 1, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 695, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 385, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 39, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 339 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-03-16T10:25:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 37, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 41, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 6, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 84, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 69, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 14, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 9, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 1, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 1497, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-03-16T10:25:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 1497, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 3, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 3, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-03-16T10:25:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 7398306, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 644, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 18.017143840, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.027976931 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 225, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 559, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 334, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 41, 2024-03-16T10:25:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 48, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 7, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 341, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 767, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 426, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 65, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 184, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 119, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 58500, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 223612 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 1476, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 392, 2024-03-16T10:25:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 617, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 188, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 701, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 31, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.093137643, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.003004440 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 29, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.030586782, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001054716 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 77, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.671157077, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.008716325 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 238, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.434823689, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001826990 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-03-16T10:25:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 30, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.014169356, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000472311 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-03-16T10:25:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 19, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.032711954, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001721681 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 74, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 1.627077714, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.021987536 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 84, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.599463630, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.007136471 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 4, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 2, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 74, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 37, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 928, 2024-03-16T10:25:09.108 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 639910, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 421, 2024-03-16T10:25:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 9952, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 193, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 1002946, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 24137, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 856, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 3388634, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 856, 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-03-16T10:25:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 856, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 26.511491645, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.030971368 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 856, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 1018 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 856, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 5, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 851, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-03-16T10:25:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 10, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 644, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 57, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 1, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 6, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-03-16T10:25:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 20, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-03-16T10:25:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 116, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 26, 2024-03-16T10:25:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 9, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 12, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 3, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 223, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 10, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 47 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 6796, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 4654497, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 6796, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 6796, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 4654497, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 856, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3388554, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 856, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3388554, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 856, 2024-03-16T10:25:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 856, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 856, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 856, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x56201762c500": { 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 1497, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 3204002, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 1497, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 644, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3204002, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:09.125 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:25:09.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:09 smithi105 ceph-mon[36559]: pgmap v501: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-03-16T10:25:09.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:09 smithi194 ceph-mon[36445]: pgmap v501: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-03-16T10:25:10.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:09 smithi039 ceph-mon[31510]: pgmap v501: 129 pgs: 3 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 113 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 219 op/s 2024-03-16T10:25:10.187 INFO:tasks.check_counter:Daemon mds.a mds.exported=9 2024-03-16T10:25:10.187 INFO:tasks.check_counter:Daemon mds.a mds.imported=7 2024-03-16T10:25:10.187 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=5 2024-03-16T10:25:10.188 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:10.188 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:10.188 DEBUG:tasks.check_counter:Getting stats from d 2024-03-16T10:25:10.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-16T10:25:10.676 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:25:11.778 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:11 smithi039 ceph-mon[31510]: pgmap v502: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 882 KiB/s wr, 133 op/s 2024-03-16T10:25:11.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:11 smithi105 ceph-mon[36559]: pgmap v502: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 882 KiB/s wr, 133 op/s 2024-03-16T10:25:11.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:11 smithi194 ceph-mon[36445]: pgmap v502: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 882 KiB/s wr, 133 op/s 2024-03-16T10:25:13.512 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:25:13.512 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:13.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1313, 2024-03-16T10:25:13.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1315, 2024-03-16T10:25:13.512 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3380856, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 417598, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.222531404, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.058328374, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.060638641, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.076229250, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1313, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.226677165, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000172640 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:13.513 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1039, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1277, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 255444, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 903211, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 8, 2024-03-16T10:25:13.514 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.233169533, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.102697762, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.041272030, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.070331873, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1274, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.273730608, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000214859 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.515 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 573440 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1488, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1444, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2353600, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 548558, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 3, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.341406507, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.124336910, 2024-03-16T10:25:13.516 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.143287072, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.081923247, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1444, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.341782897, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000236691 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1985008, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 199712 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.517 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5770, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.263553071, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000045676 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:13.518 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-03-16T10:25:13.519 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 482, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 2, 2024-03-16T10:25:13.520 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 480, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 480, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 25, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-03-16T10:25:13.521 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:13.522 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-03-16T10:25:13.523 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-03-16T10:25:13.524 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 1948, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-03-16T10:25:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 3, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 7233923, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 7233349, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.526 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 1948 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 341, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 563, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 222, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 34, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 41, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 7, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 482, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 606, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 124, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-03-16T10:25:13.527 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 52444, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215420 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.528 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.529 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.530 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.531 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-03-16T10:25:13.532 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.533 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.534 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.536 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-03-16T10:25:13.538 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 4, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 3, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3065, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:13.539 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:13.540 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:13.541 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 204487, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 420, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:13.542 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 9952, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 193, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 1230126, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 15871, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-03-16T10:25:13.543 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 16.935672934, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.004913163 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3448 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3447, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-03-16T10:25:13.544 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2132, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1313, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-03-16T10:25:13.545 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-03-16T10:25:13.546 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 117, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 27, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 5, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 7, 2024-03-16T10:25:13.547 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 2, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.548 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 3836, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 5621982, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:13.549 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 3836, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3836, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 5621982, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:13.550 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3447, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3039624, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3447, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3039624, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:13.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3447, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3447, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3447, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3447, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:13.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x5642f2f4a780": { 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:13.554 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:13.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:13.555 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.555 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:13.555 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:25:13.555 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:13 smithi039 ceph-mon[31510]: osdmap e117: 12 total, 12 up, 12 in 2024-03-16T10:25:13.555 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:13 smithi039 ceph-mon[31510]: pgmap v504: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-16T10:25:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:13 smithi105 ceph-mon[36559]: osdmap e117: 12 total, 12 up, 12 in 2024-03-16T10:25:13.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:13 smithi105 ceph-mon[36559]: pgmap v504: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-16T10:25:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:13 smithi194 ceph-mon[36445]: osdmap e117: 12 total, 12 up, 12 in 2024-03-16T10:25:13.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:13 smithi194 ceph-mon[36445]: pgmap v504: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-16T10:25:15.195 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-03-16T10:25:15.195 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-03-16T10:25:15.195 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-03-16T10:25:15.195 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:15.195 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:15.195 DEBUG:tasks.check_counter:Getting stats from g 2024-03-16T10:25:15.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-16T10:25:15.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:15 smithi039 ceph-mon[31510]: pgmap v505: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 86 KiB/s wr, 25 op/s 2024-03-16T10:25:15.698 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:25:15.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:15 smithi194 ceph-mon[36445]: pgmap v505: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 86 KiB/s wr, 25 op/s 2024-03-16T10:25:15.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:15 smithi105 ceph-mon[36559]: pgmap v505: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 86 KiB/s wr, 25 op/s 2024-03-16T10:25:17.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:17 smithi194 ceph-mon[36445]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 81 KiB/s wr, 21 op/s 2024-03-16T10:25:17.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:17 smithi039 ceph-mon[31510]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 81 KiB/s wr, 21 op/s 2024-03-16T10:25:17.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:17 smithi105 ceph-mon[36559]: pgmap v506: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 81 KiB/s wr, 21 op/s 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 4, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 4, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 2992, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 448, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.004711976, 2024-03-16T10:25:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000053295, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.001577478, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001873315, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000624438 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 3, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 238, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 480, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 275408, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.041126238, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.029946985, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.006349669, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 235, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.080051018, 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000340642 2024-03-16T10:25:18.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 275408 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 322, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 310, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1740544, 2024-03-16T10:25:18.802 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 191760, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.131358329, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.057606390, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.074354316, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.015626389, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 309, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.066387358, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000214845 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:18.803 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1740544, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 191760 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:18.804 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:18.805 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:18.806 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 147930, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 28, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:18.807 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 325, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1695128, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 325, 2024-03-16T10:25:18.808 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 325, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1695128, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:18.809 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:25:19.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:19 smithi039 ceph-mon[31510]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-03-16T10:25:19.671 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-16T10:25:19.671 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-16T10:25:19.671 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-16T10:25:19.671 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-16T10:25:19.671 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-16T10:25:19.671 DEBUG:tasks.check_counter:Getting stats from j 2024-03-16T10:25:19.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-16T10:25:19.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:19 smithi194 ceph-mon[36445]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-03-16T10:25:19.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:19 smithi105 ceph-mon[36559]: pgmap v507: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-03-16T10:25:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T10:25:21.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:21 smithi039 ceph-mon[31510]: pgmap v508: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-03-16T10:25:21.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:21 smithi105 ceph-mon[36559]: pgmap v508: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-03-16T10:25:21.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:21 smithi194 ceph-mon[36445]: pgmap v508: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-03-16T10:25:22.534 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2445, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2447, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 4913631, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 777511, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.399353288, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.108675530, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.101779499, 2024-03-16T10:25:22.535 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.147636732, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2445, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.480975817, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000196718 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T10:25:22.536 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 394, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 352, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1984494, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 202846, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 6, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.114705573, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.025809381, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.107180743, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.012448689, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 351, 2024-03-16T10:25:22.537 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.067217957, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000191504 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1983104, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 201824 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1069, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1307, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 478682, 2024-03-16T10:25:22.538 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 913890, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 8, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.219493128, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.080831711, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.039845428, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.076031496, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1305, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.229034681, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000175505 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:22.539 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 575856 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-03-16T10:25:22.540 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 5701, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.267412688, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000046906 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.541 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-03-16T10:25:22.542 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 1184, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 35, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 1149, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 1135, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-03-16T10:25:22.543 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 42, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-03-16T10:25:22.544 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:22.545 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-03-16T10:25:22.546 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:22.547 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-03-16T10:25:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 1000, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 3, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-03-16T10:25:22.549 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 8601978, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 8600175, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 1000 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.550 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 1114, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 1261, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 147, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 81, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 85, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 4, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 1184, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 1316, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 132, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-03-16T10:25:22.551 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 55804, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215420 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.552 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.554 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.555 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-03-16T10:25:22.556 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.558 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-03-16T10:25:22.559 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.560 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:22.561 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.563 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 2, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 4, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 2, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 3, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 3076, 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:22.565 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:22.566 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 250099, 2024-03-16T10:25:22.567 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 1224, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 187, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:22.568 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 3606393, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 37840, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-03-16T10:25:22.569 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3511, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3511, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3511, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 7.217860777, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002055784 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3511, 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3512 2024-03-16T10:25:22.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3511, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3511, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2168, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1341, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-03-16T10:25:22.571 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-03-16T10:25:22.572 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 117, 2024-03-16T10:25:22.573 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 27, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 7, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 10, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 3, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-03-16T10:25:22.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:22.575 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 3904, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 7002561, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 3904, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3904, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 7002561, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-03-16T10:25:22.576 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3511, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 4407679, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3511, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 4407679, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:22.577 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3511, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3511, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3511, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3511, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.579 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x564117766780": { 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-03-16T10:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-03-16T10:25:22.581 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-03-16T10:25:22.581 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-03-16T10:25:22.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-03-16T10:25:22.581 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-03-16T10:25:23.477 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-03-16T10:25:23.478 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-03-16T10:25:23.478 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-03-16T10:25:23.478 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:23.478 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:23.478 DEBUG:tasks.check_counter:Getting stats from b 2024-03-16T10:25:23.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-16T10:25:23.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:23 smithi039 ceph-mon[31510]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-03-16T10:25:23.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:23 smithi105 ceph-mon[36559]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-03-16T10:25:23.874 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:25:23.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:23 smithi194 ceph-mon[36445]: pgmap v509: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 17 op/s 2024-03-16T10:25:25.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:25 smithi105 ceph-mon[36559]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-03-16T10:25:25.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:25 smithi039 ceph-mon[31510]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-03-16T10:25:25.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:25 smithi194 ceph-mon[36445]: pgmap v510: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-03-16T10:25:26.684 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 5878, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4820, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 2373835, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 14016400, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 12, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.445038956, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.141429066, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.201442481, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.114287451, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4819, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.474548930, 2024-03-16T10:25:26.685 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000305986 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6611, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001182959, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000178 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 495344, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 155904 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 6744, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 7273, 2024-03-16T10:25:26.686 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 2937016, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 16711724, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 12, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.483065242, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.191601565, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.254099810, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.077249808, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7276, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.775455265, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000106577 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:26.687 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 11405, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002081323, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000182 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 42864, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 268464 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 9107, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 11673, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 5428097, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 10820834, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 19, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 8, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.692364178, 2024-03-16T10:25:26.688 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.258990836, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.423954220, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.103412421, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 11666, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.998380863, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000085580 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16557, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002762544, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000166 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2069488, 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 653728 2024-03-16T10:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3273, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.640681741, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000806807 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.690 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 610, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.027433052, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000044972 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 2736, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 2597, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2597, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 102.397068985, 2024-03-16T10:25:26.691 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.039428983 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 139, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 47, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 4, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 324, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 56, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 217, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 51, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 244, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 1528, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 62, 2024-03-16T10:25:26.692 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 63, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 48, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 4269, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 3382, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 135, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 39, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 100, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 793, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 29, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 1470, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 8, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 223, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:26.693 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 996, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1158534485, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 6, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 62, 2024-03-16T10:25:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 364, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 1843, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 1978, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 1275, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 109, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 644 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 39, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 135, 2024-03-16T10:25:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 4, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 161, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 122, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 35, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 8, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:26.696 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 29, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 4, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 4423, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 4423, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 10, 2024-03-16T10:25:26.697 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 10, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 33887765, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1971, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 34.599392670, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.017554232 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 239, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1776, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1537, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 99, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 132, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 33, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 326, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 2232, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 1906, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 63, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1301, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 1238, 2024-03-16T10:25:26.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 67000, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 231804 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 3915, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 877, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 2736, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 234, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 1758, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1114, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 6.838389652, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.006138590 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 244, 2024-03-16T10:25:26.700 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 10.919758819, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.044753109 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 40, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.691645360, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.017291134 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.701 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 375, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 77.869445429, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.207651854 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000229037, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000114518 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.702 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003146432, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000196652 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 37, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.632967579, 2024-03-16T10:25:26.704 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.017107231 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.329427036, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.020589189 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 463, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.205378552, 2024-03-16T10:25:26.705 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000443582 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 52, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.144480344, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002778468 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 163, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.139525583, 2024-03-16T10:25:26.706 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.019260893 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.014436475, 2024-03-16T10:25:26.707 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.007218237 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.291407414, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.129140741 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001236712, 2024-03-16T10:25:26.708 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000618356 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.709 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.021713522, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002714190 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 53, 2024-03-16T10:25:26.710 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.293881039, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005544925 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 5, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 3, 2024-03-16T10:25:26.711 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 175, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 87, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 952, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:26.712 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:26.713 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 563263, 2024-03-16T10:25:26.714 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 385, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 187, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-16T10:25:26.715 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 1260073, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 23648, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 3312, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 30240980, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 37, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 3312, 2024-03-16T10:25:26.716 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3312, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 71.150614829, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.021482673 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3312, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3598 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 3312, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 82, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 3230, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-16T10:25:26.717 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 82, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 2053, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 132, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 3, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 777, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-16T10:25:26.718 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-16T10:25:26.719 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 117, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 32, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-03-16T10:25:26.720 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 424, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 94 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-16T10:25:26.721 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 10, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 5, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 257, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 122, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 21725, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 8697104, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 21725, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:26.722 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 21725, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 8697104, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:26.723 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 3312, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 30254494, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3312, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 30254494, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 3312, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 3312, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3312, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 3312, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:26.725 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 122, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 13514, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 122, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 82, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 13514, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55c29edce500": { 2024-03-16T10:25:26.726 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 4423, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 29693461, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 4423, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1971, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 29693461, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.727 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:26.728 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-16T10:25:27.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:27 smithi105 ceph-mon[36559]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-03-16T10:25:27.698 INFO:tasks.check_counter:Daemon mds.b mds.exported=29 2024-03-16T10:25:27.698 INFO:tasks.check_counter:Daemon mds.b mds.imported=8 2024-03-16T10:25:27.698 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-16T10:25:27.698 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-03-16T10:25:27.698 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-03-16T10:25:27.698 DEBUG:tasks.check_counter:Getting stats from e 2024-03-16T10:25:27.698 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-16T10:25:27.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:27 smithi039 ceph-mon[31510]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-03-16T10:25:27.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:27 smithi194 ceph-mon[36445]: pgmap v511: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-03-16T10:25:27.964 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:25:29.745 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:29 smithi105 ceph-mon[36559]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-03-16T10:25:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:29 smithi039 ceph-mon[31510]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-03-16T10:25:29.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:29 smithi194 ceph-mon[36445]: pgmap v512: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-03-16T10:25:30.485 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-16T10:25:30.485 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:30.485 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 3327, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 3329, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 4143535, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1057993, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.244722346, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.064326482, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.064236069, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.101109046, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3327, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.112779901, 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000033898 2024-03-16T10:25:30.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 393, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 351, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1983660, 2024-03-16T10:25:30.487 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 202522, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 3, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.057816458, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.015207030, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.051624924, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.007160409, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 350, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.012543955, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000035839 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.488 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1983104, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 201824 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 8, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 247, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3740, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 668670, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.012503388, 2024-03-16T10:25:30.489 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008053959, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.002565390, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000026793, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 245, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.059346525, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000242230 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-03-16T10:25:30.490 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 668352 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5613, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.131685446, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000023460 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.491 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 329, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 11, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 318, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 313, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-16T10:25:30.493 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 7, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-16T10:25:30.494 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:30.495 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-16T10:25:30.496 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:30.497 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-03-16T10:25:30.498 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 1503, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 3, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 7401750, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 7401176, 2024-03-16T10:25:30.499 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 1503 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 208, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 534, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 326, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 17, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 28, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 11, 2024-03-16T10:25:30.500 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 329, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 699, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 370, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 51740, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-16T10:25:30.501 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:30.503 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:30.504 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.505 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.506 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.507 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.510 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-16T10:25:30.511 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-16T10:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 4, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 4, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 4480, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:30.514 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:30.515 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-16T10:25:30.516 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 192222, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 322, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:30.517 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 187, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 837466, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 19917, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-16T10:25:30.518 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 3330, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 3330, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3330, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 46.990102745, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.014111141 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3330, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3331 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.519 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 3330, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 3330, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 2054, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1274, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-16T10:25:30.520 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:30.521 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-16T10:25:30.522 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 117, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 27, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 4, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 5, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 1, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-16T10:25:30.523 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-16T10:25:30.524 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 3724, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 5772846, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 3724, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:30.525 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3724, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 5772846, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:30.526 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 3330, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 3207451, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3330, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 3207451, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.527 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 3330, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 3330, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3330, 2024-03-16T10:25:30.528 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 3330, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:30.529 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55db5c7f86e0": { 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:30.530 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:30.531 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-16T10:25:31.257 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-16T10:25:31.257 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-16T10:25:31.257 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-16T10:25:31.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:31.258 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:31.258 DEBUG:tasks.check_counter:Getting stats from h 2024-03-16T10:25:31.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-16T10:25:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:25:31.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:31 smithi039 ceph-mon[31510]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-03-16T10:25:31.756 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:31 smithi194 ceph-mon[36445]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-03-16T10:25:31.798 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:31 smithi105 ceph-mon[36559]: pgmap v513: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-03-16T10:25:33.731 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:33 smithi105 ceph-mon[36559]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2458, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2211, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1636151, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 2591199, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 11, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.185345116, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.081664526, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.109832235, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.014480190, 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:33.732 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2209, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.188872084, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000085501 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3797, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000777815, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000204 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1629, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1987, 2024-03-16T10:25:33.733 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 2615118, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1228802, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 17, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.134674777, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.048306832, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.104328778, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.007080768, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1987, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.274090879, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000137942 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:33.734 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2236, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000405227, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000181 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1998288, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 206432 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 315, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 584, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 258916, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 962952, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 14, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.032719178, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.015151589, 2024-03-16T10:25:33.735 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.015774490, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.001792142, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 581, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.078575424, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000135241 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 481, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000079805, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000165 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 676704 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.736 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 620, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.125173928, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000201893 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 20, 2024-03-16T10:25:33.737 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000534230, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000026711 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 233, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 218, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 218, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.100061731, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005046154 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 15, 2024-03-16T10:25:33.738 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 1, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 693, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 16, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 677, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 669, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 21, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 301, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 301, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 20, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 732, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 628, 2024-03-16T10:25:33.739 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 14, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 26, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 13, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 1720, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 9, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 346, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 7, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 1004, 2024-03-16T10:25:33.740 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 1125196203, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 5, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 4, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 72, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 143, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:33.741 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 198, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 65, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 16, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 44 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 37, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 28, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 3, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:33.742 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 10, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 1, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 11, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 4, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:33.743 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 714, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 714, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-03-16T10:25:33.744 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 5560350, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 250, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 6.182856051, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.024731424 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-16T10:25:33.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 644, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 645, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 44, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 45, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 699, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 723, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 24, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 301, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 598, 2024-03-16T10:25:33.746 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 297, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 56796, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 520, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 122, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 233, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 41, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 302, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 9, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003350400, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000372266 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 14, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.425264715, 2024-03-16T10:25:33.748 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.030376051 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 132, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.072082886, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000546082 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.749 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 15, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.014321525, 2024-03-16T10:25:33.752 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000954768 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 18, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.065560848, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003642269 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 20, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.433789853, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.021689492 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:33.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.085691504, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008569150 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.757 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-16T10:25:33.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 75, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 37, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 951, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:33.759 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:33.760 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 505777, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 557, 2024-03-16T10:25:33.761 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 187, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 2243158, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 27935, 2024-03-16T10:25:33.762 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 389, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 1405647, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 33, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 389, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 389, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 15.324706664, 2024-03-16T10:25:33.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.039395132 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 389, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 451 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 389, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 2, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 387, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-03-16T10:25:33.764 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 251, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 59, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 17, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:33.765 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-16T10:25:33.766 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 117, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 27, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 10, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 16, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 6, 2024-03-16T10:25:33.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 71, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 20 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-16T10:25:33.768 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 5, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 1, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 5, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 1, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 4398, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 4089508, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 4398, 2024-03-16T10:25:33.769 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4398, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4089508, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:33.770 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 389, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1405780, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 389, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1405780, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:33.771 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 389, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 389, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 389, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 389, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:33.772 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 149, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 149, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55e298dcc500": { 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:33.773 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 714, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1366046, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 714, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 250, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1366046, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:33.774 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-16T10:25:33.775 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:33 smithi039 ceph-mon[31510]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-03-16T10:25:33.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:33 smithi194 ceph-mon[36445]: pgmap v514: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-03-16T10:25:34.511 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-16T10:25:34.511 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-03-16T10:25:34.511 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=5 2024-03-16T10:25:34.511 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:34.511 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:34.511 DEBUG:tasks.check_counter:Getting stats from k 2024-03-16T10:25:34.511 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-16T10:25:34.766 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T10:25:35.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:35 smithi039 ceph-mon[31510]: pgmap v515: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-16T10:25:35.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:35 smithi105 ceph-mon[36559]: pgmap v515: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-16T10:25:35.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:35 smithi194 ceph-mon[36445]: pgmap v515: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 0, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 0, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 0, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001145234, 2024-03-16T10:25:36.634 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.000329418, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T10:25:36.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 333, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 321, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1743536, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 198800, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.051350658, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.014243299, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.039915286, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008203306, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 320, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.012400394, 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000038751 2024-03-16T10:25:36.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1743536, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 198800 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 8, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 251, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3632, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 355632, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-03-16T10:25:36.637 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.013028509, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008606402, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.002323372, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 248, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022176111, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000089419 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.638 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3632, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 355632 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:36.639 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:36.640 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 147948, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 29, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-16T10:25:36.641 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-16T10:25:36.642 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 337, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1696585, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 337, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 337, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1696585, 2024-03-16T10:25:36.643 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-16T10:25:36.644 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-16T10:25:37.345 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-16T10:25:37.345 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-16T10:25:37.345 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-16T10:25:37.345 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-16T10:25:37.345 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-16T10:25:37.346 DEBUG:tasks.check_counter:Getting stats from c 2024-03-16T10:25:37.346 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-16T10:25:37.614 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:37 smithi105 ceph-mon[36559]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-16T10:25:37.732 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:25:37.861 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:37 smithi194 ceph-mon[36445]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-16T10:25:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:37 smithi039 ceph-mon[31510]: pgmap v516: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-03-16T10:25:39.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:39 smithi105 ceph-mon[36559]: pgmap v517: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-03-16T10:25:39.934 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:39 smithi194 ceph-mon[36445]: pgmap v517: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-03-16T10:25:39.934 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T10:25:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:39.934 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 612, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 813, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 178139, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 5175325, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 20, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.083091209, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.034181178, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.019073754, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.022934521, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 809, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.106189945, 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000131260 2024-03-16T10:25:39.935 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 24, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000003048, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000127 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 6016, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 443504 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1598, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2304, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 961037, 2024-03-16T10:25:39.936 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 2284528, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 11, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.122385046, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.049457235, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.093572758, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.001097053, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2304, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.262200652, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000113802 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3168, 2024-03-16T10:25:39.937 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000617466, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000194 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 221360 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 3002, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2960, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4034604, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1937731, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 7, 2024-03-16T10:25:39.938 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.237539842, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.080417570, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.178693913, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.013293742, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2959, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.298898252, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000101013 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4713, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001492194, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000316 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.939 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2005088, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 217568 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 922, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.363330189, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000394067 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.940 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 85, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003745185, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000044061 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 503, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 454, 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T10:25:39.941 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 454, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 4.188183989, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.009225074 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 49, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 5, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 19, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 1, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 1180, 2024-03-16T10:25:39.942 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 76, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 965, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 139, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 1071, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 145, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 519, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 519, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 50, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 1127, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 978, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 23, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 21, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:39.943 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 18, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 588, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 3, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 132, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 15, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 936, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 1, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 1004, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 1125196203, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 5, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:39.944 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 4, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 100, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 273, 2024-03-16T10:25:39.945 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 391, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 183, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 22, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 142 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 53, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 15, 2024-03-16T10:25:39.946 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 1, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 25, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 16, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 9, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 3, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:39.947 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 3, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 1, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 1004, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 1004, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T10:25:39.948 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 3, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 3, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 8621864, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 398, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 6.868364992, 2024-03-16T10:25:39.949 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.017257198 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 1136, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 1280, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 144, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 103, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 108, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 5, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 1182, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 1346, 2024-03-16T10:25:39.950 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 164, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 519, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 768, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 249, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 60328, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223612 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 742, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 89, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 503, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 59, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 210, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9, 2024-03-16T10:25:39.951 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.025093522, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002788169 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 8, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004668100, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000583512 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T10:25:39.952 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 10, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.132097275, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.013209727 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 330, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 3.524193329, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.010679373 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.953 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.954 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T10:25:39.955 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.956 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 17, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.013506450, 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000794497 2024-03-16T10:25:39.957 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 42, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.051078028, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001216143 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 17, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.299053529, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.017591384 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.958 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:39.959 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.961 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 21, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.138493756, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.006594940 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.963 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 4, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 2, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 119, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 59, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 960, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T10:25:39.964 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:39.965 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:39.966 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 487239, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 1085, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 176, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 2, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T10:25:39.967 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 187, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 3914783, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 41544, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T10:25:39.968 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 647, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 4537650, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 33, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 647, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 647, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 13.149738687, 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.020324171 2024-03-16T10:25:39.969 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 647, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 766 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 647, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 14, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 633, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 9, 2024-03-16T10:25:39.970 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 407, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 71, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 49, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:39.971 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T10:25:39.972 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 117, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 27, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 11, 2024-03-16T10:25:39.973 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 11, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 147, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 10, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 43 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.974 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 10, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 5, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 35, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 16, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:39.975 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 5208, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 4679476, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 5208, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 5208, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 4679476, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.976 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 647, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 4539858, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 647, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 4539858, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 647, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 647, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 647, 2024-03-16T10:25:39.978 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 647, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 16, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2288, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 16, 2024-03-16T10:25:39.979 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 9, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2288, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55d5aba56500": { 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1004, 2024-03-16T10:25:39.980 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 4427560, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1004, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 398, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 4427560, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:39.981 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T10:25:40.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:39 smithi039 ceph-mon[31510]: pgmap v517: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-03-16T10:25:40.724 INFO:tasks.check_counter:Daemon mds.c mds.exported=3 2024-03-16T10:25:40.724 INFO:tasks.check_counter:Daemon mds.c mds.imported=15 2024-03-16T10:25:40.724 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=5 2024-03-16T10:25:40.724 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:40.724 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:40.724 DEBUG:tasks.check_counter:Getting stats from f 2024-03-16T10:25:40.724 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-16T10:25:41.011 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:25:41.705 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:41 smithi194 ceph-mon[36445]: pgmap v518: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-16T10:25:41.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:41 smithi105 ceph-mon[36559]: pgmap v518: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-16T10:25:42.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:41 smithi039 ceph-mon[31510]: pgmap v518: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1580, 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1580, 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 8882957, 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 501626, 2024-03-16T10:25:42.984 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.136847961, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.036624050, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.041920334, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.047449981, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1579, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.061963912, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000039242 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1791, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1748, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 15450613, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 647100, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.177915612, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.045718111, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.083625186, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.054391392, 2024-03-16T10:25:42.986 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1747, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.108215576, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000061943 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1984144, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 203424 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 753, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 996, 2024-03-16T10:25:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 8579221, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 987687, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 9, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.076815304, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.026099905, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.024232142, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.020225430, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 992, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.072333959, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000072917 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 640, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 749504 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 6279, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.144929579, 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000023081 2024-03-16T10:25:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T10:25:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 396, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 25, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 371, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:42.991 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 361, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 17, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-03-16T10:25:42.992 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 996, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 1158534485, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 6, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:42.993 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T10:25:42.994 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:42.995 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 4427, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T10:25:42.996 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 10, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 34084277, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 34034660, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 4427 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T10:25:42.997 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 191, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 1740, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 1549, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 45, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 89, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 44, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 396, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 1860, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 1464, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 52708, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215420 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T10:25:42.998 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:42.999 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.001 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.002 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:43.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:43.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.007 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:43.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 5, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 3, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 3, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 3629, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.009 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:43.010 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:43.011 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 218731, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 396, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:43.012 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 187, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 1028231, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 18321, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 3720, 2024-03-16T10:25:43.013 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 26, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 4, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 3720, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3720, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 47.376060533, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.012735500 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3720, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 3721 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 3720, 2024-03-16T10:25:43.014 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 3720, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 2192, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1525, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:43.015 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 2, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T10:25:43.016 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 117, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T10:25:43.017 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 27, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 7, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 7, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T10:25:43.018 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 4120, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 32518817, 2024-03-16T10:25:43.019 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 4120, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 4120, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 32518817, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:43.020 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3720, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 29889983, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3720, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 29889983, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.021 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3720, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3720, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3720, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3720, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:43.023 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x557eee29c6e0": { 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T10:25:43.024 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:43.025 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T10:25:43.779 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:43 smithi194 ceph-mon[36445]: pgmap v519: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-16T10:25:43.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:43 smithi039 ceph-mon[31510]: pgmap v519: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-16T10:25:43.808 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-16T10:25:43.809 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-16T10:25:43.809 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-03-16T10:25:43.809 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:43.809 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:43.809 DEBUG:tasks.check_counter:Getting stats from i 2024-03-16T10:25:43.809 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-16T10:25:43.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:43 smithi105 ceph-mon[36559]: pgmap v519: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-03-16T10:25:44.074 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:25:45.627 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:45 smithi194 ceph-mon[36445]: pgmap v520: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-16T10:25:45.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:45 smithi105 ceph-mon[36559]: pgmap v520: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 5714, 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 5256, 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4534177, 2024-03-16T10:25:46.003 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 5513372, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 10, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.372962721, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.137052619, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.256626457, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.032333425, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5255, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.624201679, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000118782 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:46.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 8428, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001499119, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000177 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2007296, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 208288 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 512, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 751, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 331160, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1123027, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 14, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 9, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.054236074, 2024-03-16T10:25:46.005 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.021829250, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.021813473, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.009484281, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 748, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.103025095, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000137734 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 466, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000152657, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000327 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1408, 2024-03-16T10:25:46.006 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 675520 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2421, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 3274, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1163370, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1914186, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 16, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.183810086, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.071751694, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.124255708, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.008625796, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3273, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.356066379, 2024-03-16T10:25:46.007 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000108788 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4201, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000824279, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000196 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 8048, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 19600 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.008 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1073, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.489439870, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000456141 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 520, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.023053240, 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000044333 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.009 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 672, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 666, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 666, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 48.962500804, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.073517268 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 6, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 2, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 35, 2024-03-16T10:25:46.010 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 3, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 389, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 118, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 260, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 11, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 219, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 283, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 118, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 118, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 37, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 1967, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 1684, 2024-03-16T10:25:46.011 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 2, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 71, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 23, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 218, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 4, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 123, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 6, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 230, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 9, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 1004, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 1125196203, 2024-03-16T10:25:46.012 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 5, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 67, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 296, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 595, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:46.013 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 853, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 420, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 87, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 463 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 81, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 28, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 5, 2024-03-16T10:25:46.014 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 111, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 106, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 3, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:46.015 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 5, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 3, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 1995, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 1995, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 3, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T10:25:46.016 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 3, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 7262651, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 408, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 5.122557755, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.012555288 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.017 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 364, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 673, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 309, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 68, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 73, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 5, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 394, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 785, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 391, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 118, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 424, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 306, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 57696, 2024-03-16T10:25:46.018 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223612 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 1425, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 209, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 672, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 43, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 655, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 104, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.211458170, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002033251 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 39, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.470265035, 2024-03-16T10:25:46.019 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.012058077 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.053188540, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.010637708 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 262, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 47.089102431, 2024-03-16T10:25:46.020 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.179729398 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.021 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T10:25:46.022 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.023 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:45 smithi039 ceph-mon[31510]: pgmap v520: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-03-16T10:25:46.023 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.023 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.024 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 112, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.122511961, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001093856 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 27, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.054240562, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002008909 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 110, 2024-03-16T10:25:46.025 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.935637643, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.008505796 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.026 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T10:25:46.027 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T10:25:46.028 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T10:25:46.029 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 7, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.026096462, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003728066 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T10:25:46.030 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 4, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 2, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 74, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 37, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 971, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:46.031 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:46.032 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:46.033 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 484977, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 342, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T10:25:46.034 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 187, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 1343759, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 17107, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:46.035 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 1090, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 3338192, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 28, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 1090, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1090, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 20.137598141, 2024-03-16T10:25:46.036 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.018474860 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1090, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1302 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 1090, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 74, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 1016, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 72, 2024-03-16T10:25:46.037 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 480, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 118, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 227, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:46.038 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T10:25:46.039 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 117, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 27, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2024-03-16T10:25:46.040 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 291, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 4, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 80 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 1 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.041 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 10, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 5, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 212, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 106, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:46.042 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 8643, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 5225192, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 8643, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 8643, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 5225192, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.043 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 1090, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3349978, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1090, 2024-03-16T10:25:46.044 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3349978, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:46.045 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 1090, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1090, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1090, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1090, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:46.046 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 106, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 11818, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 106, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 72, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 11818, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:46.047 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x5639e1820500": { 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1995, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 3068347, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1995, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 408, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3068347, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:46.048 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:46.049 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.049 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:46.049 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T10:25:47.011 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-03-16T10:25:47.011 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-03-16T10:25:47.011 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=5 2024-03-16T10:25:47.011 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:47.011 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:47.011 DEBUG:tasks.check_counter:Getting stats from l 2024-03-16T10:25:47.012 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 65a28618-e37b-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-16T10:25:47.263 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T10:25:47.698 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:47 smithi194 ceph-mon[36445]: pgmap v521: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:47.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:47 smithi105 ceph-mon[36559]: pgmap v521: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:48.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:47 smithi039 ceph-mon[31510]: pgmap v521: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:49.172 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T10:25:49.172 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1065, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1064, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 360359, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 337529, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.099521093, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.027875828, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.028512393, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.034157062, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1063, 2024-03-16T10:25:49.173 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.055502684, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000052213 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2496, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2454, 2024-03-16T10:25:49.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 3884286, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 873689, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.190825238, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.055053375, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.081326302, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.057665879, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2453, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.075901296, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000030942 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:49.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 1985616, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 207104 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 6, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 253, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1828, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 674773, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 9, 2024-03-16T10:25:49.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.013085287, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.008580931, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.002362130, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000064110, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 250, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.049146864, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000196587 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 673808 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5372, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.132919973, 2024-03-16T10:25:49.178 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000024743 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-03-16T10:25:49.179 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 671, 2024-03-16T10:25:49.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 9, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 662, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 658, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 11, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T10:25:49.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-03-16T10:25:49.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T10:25:49.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T10:25:49.184 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-03-16T10:25:49.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 714, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T10:25:49.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 5560350, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 5559776, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.187 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 714 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 615, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 620, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 5, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 19, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 19, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 671, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 671, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-03-16T10:25:49.188 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 52104, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215420 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T10:25:49.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.193 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T10:25:49.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T10:25:49.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T10:25:49.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T10:25:49.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.200 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 2, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 2, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 2, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 2756, 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T10:25:49.201 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T10:25:49.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T10:25:49.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 221958, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 697, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T10:25:49.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9712, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 187, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 1971271, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 24283, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T10:25:49.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 3159, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 3159, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3159, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 89.988209259, 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.028486296 2024-03-16T10:25:49.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3159, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 3160 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 3159, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 3159, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 1961, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1196, 2024-03-16T10:25:49.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T10:25:49.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T10:25:49.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 117, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 27, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 6, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 8, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 2, 2024-03-16T10:25:49.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.211 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 3563, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 3902406, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 3563, 2024-03-16T10:25:49.212 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3563, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3902406, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:49.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3159, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1366051, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3159, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1366051, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T10:25:49.214 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3159, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3159, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3159, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3159, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.216 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x5560a7e346e0": { 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T10:25:49.217 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T10:25:49.218 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T10:25:49.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:49 smithi105 ceph-mon[36559]: pgmap v522: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:49.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:49 smithi194 ceph-mon[36445]: pgmap v522: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:50.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:49 smithi039 ceph-mon[31510]: pgmap v522: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:50.047 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-16T10:25:50.047 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-16T10:25:50.047 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-16T10:25:50.047 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-16T10:25:50.047 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-16T10:25:50.047 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-16T10:25:50.058 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T10:25:50.058 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:25:50.058 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:25:50.132 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-03-16T10:25:50.132 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:25:50.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:25:50.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 -- 172.21.15.39:0/1807826339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f6410bb50 msgr2=0x7f8f6410dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 --2- 172.21.15.39:0/1807826339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f6410bb50 0x7f8f6410dff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009ca0 tx=0x7f8f5c02c180 comp rx=0 tx=0).stop 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 -- 172.21.15.39:0/1807826339 shutdown_connections 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 --2- 172.21.15.39:0/1807826339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f6410bb50 0x7f8f6410dff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 --2- 172.21.15.39:0/1807826339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64103050 0x7f8f6410b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 --2- 172.21.15.39:0/1807826339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f64102730 0x7f8f64102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 -- 172.21.15.39:0/1807826339 >> 172.21.15.39:0/1807826339 conn(0x7f8f640fbf70 msgr2=0x7f8f640fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 -- 172.21.15.39:0/1807826339 shutdown_connections 2024-03-16T10:25:50.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.414+0000 7f8f6a200640 1 -- 172.21.15.39:0/1807826339 wait complete. 2024-03-16T10:25:50.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 Processor -- start 2024-03-16T10:25:50.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 -- start start 2024-03-16T10:25:50.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 0x7f8f6419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6410bb50 0x7f8f6419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111910 con 0x7f8f6410bb50 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111790 con 0x7f8f64103050 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.415+0000 7f8f6a200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111a90 con 0x7f8f64102730 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 0x7f8f6419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56768/0 (socket says 172.21.15.39:56768) 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 -- 172.21.15.39:0/2917307487 learned_addr learned my addr 172.21.15.39:0/2917307487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f63e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 0x7f8f6419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36018/0 (socket says 172.21.15.39:36018) 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f69800640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6410bb50 0x7f8f6419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 -- 172.21.15.39:0/2917307487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 msgr2=0x7f8f6419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 0x7f8f6419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 -- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6410bb50 msgr2=0x7f8f6419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6410bb50 0x7f8f6419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 -- 172.21.15.39:0/2917307487 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6419ef60 con 0x7f8f64102730 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.416+0000 7f8f68e00640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9f0 tx=0x7f8f5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.417+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800cd80 con 0x7f8f64102730 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.417+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58010910 con 0x7f8f64102730 2024-03-16T10:25:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.417+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6419f250 con 0x7f8f64102730 2024-03-16T10:25:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.417+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58018990 con 0x7f8f64102730 2024-03-16T10:25:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.417+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641a6ae0 con 0x7f8f64102730 2024-03-16T10:25:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.418+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f640684f0 con 0x7f8f64102730 2024-03-16T10:25:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.420+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f8f58010ac0 con 0x7f8f64102730 2024-03-16T10:25:50.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.421+0000 7f8f61600640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 0x7f8f4c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.421+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f8f580cf7b0 con 0x7f8f64102730 2024-03-16T10:25:50.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.421+0000 7f8f63e00640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 0x7f8f4c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.422+0000 7f8f63e00640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 0x7f8f4c0a4e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f6410d360 tx=0x7f8f50005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.423+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8f5809ba50 con 0x7f8f64102730 2024-03-16T10:25:50.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.719+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f641a6d90 con 0x7f8f64102730 2024-03-16T10:25:50.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.720+0000 7f8f61600640 1 -- 172.21.15.39:0/2917307487 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+25415 (secure 0 0 0) 0x7f8f5809faf0 con 0x7f8f64102730 2024-03-16T10:25:50.721 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:50.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:25:12.293141+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3265323877":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/2375807750":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/332905917":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/2792410890":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:25:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 msgr2=0x7f8f4c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 0x7f8f4c0a4e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f6410d360 tx=0x7f8f50005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 msgr2=0x7f8f6419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800e9f0 tx=0x7f8f5800eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 shutdown_connections 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6410bb50 0x7f8f6419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f64103050 0x7f8f6419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f8f4c0a2970 0x7f8f4c0a4e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 --2- 172.21.15.39:0/2917307487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f64102730 0x7f8f6419a010 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 >> 172.21.15.39:0/2917307487 conn(0x7f8f640fbf70 msgr2=0x7f8f6410dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 shutdown_connections 2024-03-16T10:25:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.724+0000 7f8f6a200640 1 -- 172.21.15.39:0/2917307487 wait complete. 2024-03-16T10:25:50.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:00:56.748725+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:06:01.836087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:06:03.184357+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-16T10:06:06.832116+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-16T10:25:50.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:25:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 -- 172.21.15.39:0/31849623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0104340 msgr2=0x7fc2d0110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 --2- 172.21.15.39:0/31849623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0104340 0x7fc2d0110850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009ca0 tx=0x7fc2c802c0f0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 -- 172.21.15.39:0/31849623 shutdown_connections 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 --2- 172.21.15.39:0/31849623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0104340 0x7fc2d0110850 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 --2- 172.21.15.39:0/31849623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0103980 0x7fc2d0103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 --2- 172.21.15.39:0/31849623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0109990 0x7fc2d0109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 -- 172.21.15.39:0/31849623 >> 172.21.15.39:0/31849623 conn(0x7fc2d00ff490 msgr2=0x7fc2d01018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 -- 172.21.15.39:0/31849623 shutdown_connections 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.971+0000 7fc2d5200640 1 -- 172.21.15.39:0/31849623 wait complete. 2024-03-16T10:25:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.972+0000 7fc2d5200640 1 Processor -- start 2024-03-16T10:25:50.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 -- start start 2024-03-16T10:25:50.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0104340 0x7fc2d019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0113da0 con 0x7fc2d0104340 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0113c20 con 0x7fc2d0103980 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2d5200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d0113f20 con 0x7fc2d0109990 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2cde00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2ce800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56788/0 (socket says 172.21.15.39:56788) 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.973+0000 7fc2cde00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36028/0 (socket says 172.21.15.39:36028) 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 -- 172.21.15.39:0/2442751265 learned_addr learned my addr 172.21.15.39:0/2442751265 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:50.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cd400640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0104340 0x7fc2d019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 -- 172.21.15.39:0/2442751265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 msgr2=0x7fc2d019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 -- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0104340 msgr2=0x7fc2d019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0104340 0x7fc2d019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 -- 172.21.15.39:0/2442751265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d019f1f0 con 0x7fc2d0103980 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2ce800640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2cde00640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e9f0 tx=0x7fc2c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000cd80 con 0x7fc2d0103980 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c0010910 con 0x7fc2d0103980 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.974+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0018990 con 0x7fc2d0103980 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.975+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d019f4e0 con 0x7fc2d0103980 2024-03-16T10:25:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.975+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d01a6d70 con 0x7fc2d0103980 2024-03-16T10:25:50.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.976+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d0068a20 con 0x7fc2d0103980 2024-03-16T10:25:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.978+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7fc2c0010ac0 con 0x7fc2d0103980 2024-03-16T10:25:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.979+0000 7fc2c6a00640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 0x7fc2b40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.979+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7fc2c00cf4e0 con 0x7fc2d0103980 2024-03-16T10:25:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.979+0000 7fc2cd400640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 0x7fc2b40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:50.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.980+0000 7fc2cd400640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 0x7fc2b40a4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0095f0 tx=0x7fc2bc005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:50.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:50.986+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc2c009b780 con 0x7fc2d0103980 2024-03-16T10:25:51.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.191+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2d002d1b0 con 0x7fc2d0103980 2024-03-16T10:25:51.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.192+0000 7fc2c6a00640 1 -- 172.21.15.39:0/2442751265 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) ==== 93+0+10 (secure 0 0 0) 0x7fc2c009f820 con 0x7fc2d0103980 2024-03-16T10:25:51.193 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-03-16T10:25:51.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 msgr2=0x7fc2b40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 0x7fc2b40a4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0095f0 tx=0x7fc2bc005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:51.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 msgr2=0x7fc2d019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e9f0 tx=0x7fc2c000eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 shutdown_connections 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc2b40a29f0 0x7fc2b40a4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2d0109990 0x7fc2d019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2d0104340 0x7fc2d019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 --2- 172.21.15.39:0/2442751265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2d0103980 0x7fc2d019a200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 >> 172.21.15.39:0/2442751265 conn(0x7fc2d00ff490 msgr2=0x7fc2d00ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 shutdown_connections 2024-03-16T10:25:51.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.195+0000 7fc2d5200640 1 -- 172.21.15.39:0/2442751265 wait complete. 2024-03-16T10:25:51.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:25:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 -- 172.21.15.39:0/1649399502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 msgr2=0x7f7824107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 --2- 172.21.15.39:0/1649399502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f7824107a40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7818009900 tx=0x7f781802bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 -- 172.21.15.39:0/1649399502 shutdown_connections 2024-03-16T10:25:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 --2- 172.21.15.39:0/1649399502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824102010 0x7f782410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 --2- 172.21.15.39:0/1649399502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824101650 0x7f7824101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 --2- 172.21.15.39:0/1649399502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f7824107a40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 -- 172.21.15.39:0/1649399502 >> 172.21.15.39:0/1649399502 conn(0x7f78240fd140 msgr2=0x7f78240ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 -- 172.21.15.39:0/1649399502 shutdown_connections 2024-03-16T10:25:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.459+0000 7f782aa00640 1 -- 172.21.15.39:0/1649399502 wait complete. 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 Processor -- start 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 -- start start 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824101650 0x7f782419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f782419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7824111a80 con 0x7f7824101650 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7824111900 con 0x7f7824107660 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.460+0000 7f782aa00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7824111c00 con 0x7f7824102010 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56814/0 (socket says 172.21.15.39:56814) 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f782a000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f782419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 -- 172.21.15.39:0/1199504569 learned_addr learned my addr 172.21.15.39:0/1199504569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7829600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824101650 0x7f782419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 -- 172.21.15.39:0/1199504569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 msgr2=0x7f782419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f782419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 -- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824101650 msgr2=0x7f782419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824101650 0x7f782419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 -- 172.21.15.39:0/1199504569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782419f0f0 con 0x7f7824102010 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.461+0000 7f7828c00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f781000ea60 tx=0x7f781000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.462+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781000cde0 con 0x7f7824102010 2024-03-16T10:25:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.462+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78100035f0 con 0x7f7824102010 2024-03-16T10:25:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.462+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810010670 con 0x7f7824102010 2024-03-16T10:25:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.462+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782419f3e0 con 0x7f7824102010 2024-03-16T10:25:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.462+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241a6c70 con 0x7f7824102010 2024-03-16T10:25:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.463+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7824068940 con 0x7f7824102010 2024-03-16T10:25:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.465+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f78100037a0 con 0x7f7824102010 2024-03-16T10:25:51.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.466+0000 7f7822000640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 0x7f780c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.466+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f78100cf520 con 0x7f7824102010 2024-03-16T10:25:51.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.466+0000 7f7829600640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 0x7f780c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.467+0000 7f7829600640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 0x7f780c0a4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f781802efd0 tx=0x7f7818000bc0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:51.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.468+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f781009be10 con 0x7f7824102010 2024-03-16T10:25:51.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.676+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f78241a6ed0 con 0x7f7824102010 2024-03-16T10:25:51.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.677+0000 7f7822000640 1 -- 172.21.15.39:0/1199504569 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) ==== 104+0+11 (secure 0 0 0) 0x7f781009b710 con 0x7f7824102010 2024-03-16T10:25:51.677 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:25:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.679+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 msgr2=0x7f780c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.679+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 0x7f780c0a4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f781802efd0 tx=0x7f7818000bc0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.679+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 msgr2=0x7f782419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.679+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f781000ea60 tx=0x7f781000ef20 comp rx=0 tx=0).stop 2024-03-16T10:25:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 shutdown_connections 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7824107660 0x7f782419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f780c0a29f0 0x7f780c0a4eb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7824102010 0x7f782419a640 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 --2- 172.21.15.39:0/1199504569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7824101650 0x7f782419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 >> 172.21.15.39:0/1199504569 conn(0x7f78240fd140 msgr2=0x7f782410ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 shutdown_connections 2024-03-16T10:25:51.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.680+0000 7f782aa00640 1 -- 172.21.15.39:0/1199504569 wait complete. 2024-03-16T10:25:51.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:25:51.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:51 smithi105 ceph-mon[36559]: pgmap v523: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:51.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2917307487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:51.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:51 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2442751265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 -- 172.21.15.39:0/1052452827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 msgr2=0x7f1240102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 --2- 172.21.15.39:0/1052452827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240102bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1230009900 tx=0x7f123002bcc0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 -- 172.21.15.39:0/1052452827 shutdown_connections 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 --2- 172.21.15.39:0/1052452827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f124010bc30 0x7f124010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 --2- 172.21.15.39:0/1052452827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1240103130 0x7f124010b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 --2- 172.21.15.39:0/1052452827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240102bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 -- 172.21.15.39:0/1052452827 >> 172.21.15.39:0/1052452827 conn(0x7f12400fc030 msgr2=0x7f12400fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 -- 172.21.15.39:0/1052452827 shutdown_connections 2024-03-16T10:25:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.930+0000 7f1245c00640 1 -- 172.21.15.39:0/1052452827 wait complete. 2024-03-16T10:25:51.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.931+0000 7f1245c00640 1 Processor -- start 2024-03-16T10:25:51.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.931+0000 7f1245c00640 1 -- start start 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 0x7f12401009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f124010bc30 0x7f1240100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1240111a80 con 0x7f1240103130 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1240111900 con 0x7f1240102810 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245c00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1240111c00 con 0x7f124010bc30 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 0x7f12401009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123fe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36060/0 (socket says 172.21.15.39:36060) 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123fe00640 1 -- 172.21.15.39:0/1850989289 learned_addr learned my addr 172.21.15.39:0/1850989289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 0x7f12401009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51620/0 (socket says 172.21.15.39:51620) 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f1245200640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f124010bc30 0x7f1240100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.932+0000 7f123fe00640 1 -- 172.21.15.39:0/1850989289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f124010bc30 msgr2=0x7f1240100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123fe00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f124010bc30 0x7f1240100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123fe00640 1 -- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 msgr2=0x7f12401009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:51.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123fe00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 0x7f12401009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:51.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123fe00640 1 -- 172.21.15.39:0/1850989289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401a6400 con 0x7f1240102810 2024-03-16T10:25:51.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123fe00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f12300043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230042020 con 0x7f1240102810 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1230005000 con 0x7f1240102810 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12401a6690 con 0x7f1240102810 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230003b20 con 0x7f1240102810 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.933+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a6be0 con 0x7f1240102810 2024-03-16T10:25:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.935+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12400684f0 con 0x7f1240102810 2024-03-16T10:25:51.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.936+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f1230045020 con 0x7f1240102810 2024-03-16T10:25:51.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.938+0000 7f123cc00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 0x7f12240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:51.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.938+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f12300f0e70 con 0x7f1240102810 2024-03-16T10:25:51.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.938+0000 7f123f400640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 0x7f12240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.939+0000 7f123f400640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 0x7f12240a4e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f122c0095f0 tx=0x7f122c005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:51.946+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f123004c020 con 0x7f1240102810 2024-03-16T10:25:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:51 smithi194 ceph-mon[36445]: pgmap v523: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2917307487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:51.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:51 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2442751265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:51 smithi039 ceph-mon[31510]: pgmap v523: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2917307487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:52.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:51 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2442751265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.168+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1240106cd0 con 0x7f1240102810 2024-03-16T10:25:52.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.169+0000 7f123cc00640 1 -- 172.21.15.39:0/1850989289 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) ==== 100+0+11 (secure 0 0 0) 0x7f12300bd870 con 0x7f1240102810 2024-03-16T10:25:52.169 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-03-16T10:25:52.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 msgr2=0x7f12240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 0x7f12240a4e60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f122c0095f0 tx=0x7f122c005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:52.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 msgr2=0x7f1240100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1230000c00 tx=0x7f12300043c0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 shutdown_connections 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f12240a29a0 0x7f12240a4e60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f124010bc30 0x7f1240100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1240103130 0x7f12401009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 --2- 172.21.15.39:0/1850989289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1240102810 0x7f1240100460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 >> 172.21.15.39:0/1850989289 conn(0x7f12400fc030 msgr2=0x7f12400fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 shutdown_connections 2024-03-16T10:25:52.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.172+0000 7f1245c00640 1 -- 172.21.15.39:0/1850989289 wait complete. 2024-03-16T10:25:52.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 -- 172.21.15.39:0/3680767697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 msgr2=0x7fdb00101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 --2- 172.21.15.39:0/3680767697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb00101bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 -- 172.21.15.39:0/3680767697 shutdown_connections 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 --2- 172.21.15.39:0/3680767697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 0x7fdb0010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 --2- 172.21.15.39:0/3680767697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb00101bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 --2- 172.21.15.39:0/3680767697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb00107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 -- 172.21.15.39:0/3680767697 >> 172.21.15.39:0/3680767697 conn(0x7fdb000fd240 msgr2=0x7fdb000ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 -- 172.21.15.39:0/3680767697 shutdown_connections 2024-03-16T10:25:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.451+0000 7fdb07000640 1 -- 172.21.15.39:0/3680767697 wait complete. 2024-03-16T10:25:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 Processor -- start 2024-03-16T10:25:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 -- start start 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 0x7fdb0019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00111bc0 con 0x7fdb00102110 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00111a40 con 0x7fdb00101750 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdb07000640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00111d40 con 0x7fdb00107760 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.453+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56852/0 (socket says 172.21.15.39:56852) 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdaff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 0x7fdb0019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 -- 172.21.15.39:0/3210775358 learned_addr learned my addr 172.21.15.39:0/3210775358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36090/0 (socket says 172.21.15.39:36090) 2024-03-16T10:25:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 -- 172.21.15.39:0/3210775358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 msgr2=0x7fdb0019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 -- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 msgr2=0x7fdb0019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 0x7fdb0019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 -- 172.21.15.39:0/3210775358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0019f290 con 0x7fdb00107760 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdaffe00640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.454+0000 7fdb05000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eaa0 tx=0x7fdaf800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.455+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800cdb0 con 0x7fdb00107760 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.455+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0019f580 con 0x7fdb00107760 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.455+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf8005020 con 0x7fdb00107760 2024-03-16T10:25:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.455+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80116b0 con 0x7fdb00107760 2024-03-16T10:25:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.455+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001a6e10 con 0x7fdb00107760 2024-03-16T10:25:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.456+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb00068a20 con 0x7fdb00107760 2024-03-16T10:25:52.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.458+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7fdaf8005560 con 0x7fdb00107760 2024-03-16T10:25:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.459+0000 7fdafcc00640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 0x7fdae40a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.459+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7fdaf80cf4a0 con 0x7fdb00107760 2024-03-16T10:25:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.460+0000 7fdaffe00640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 0x7fdae40a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.460+0000 7fdaffe00640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 0x7fdae40a4e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009f40 tx=0x7fdaf0005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:52.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.468+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdaf8016030 con 0x7fdb00107760 2024-03-16T10:25:52.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.737+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdb001a70b0 con 0x7fdb00107760 2024-03-16T10:25:52.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.737+0000 7fdafcc00640 1 -- 172.21.15.39:0/3210775358 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v117) ==== 103+0+11 (secure 0 0 0) 0x7fdaf809b740 con 0x7fdb00107760 2024-03-16T10:25:52.738 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-03-16T10:25:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 msgr2=0x7fdae40a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 0x7fdae40a4e30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009f40 tx=0x7fdaf0005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 msgr2=0x7fdb0019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800eaa0 tx=0x7fdaf800ef60 comp rx=0 tx=0).stop 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 shutdown_connections 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fdae40a2970 0x7fdae40a4e30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb00107760 0x7fdb0019eb60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb00102110 0x7fdb0019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 --2- 172.21.15.39:0/3210775358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb00101750 0x7fdb0019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 >> 172.21.15.39:0/3210775358 conn(0x7fdb000fd240 msgr2=0x7fdb0010cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 shutdown_connections 2024-03-16T10:25:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.740+0000 7fdb07000640 1 -- 172.21.15.39:0/3210775358 wait complete. 2024-03-16T10:25:52.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:25:52.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1199504569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:52 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1850989289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1199504569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:52 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1850989289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1199504569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:52 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1850989289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T10:25:52.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 -- 172.21.15.39:0/256807860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb281003f0 msgr2=0x7fbb281007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 --2- 172.21.15.39:0/256807860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb281003f0 0x7fbb281007d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1802bc50 comp rx=0 tx=0).stop 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 -- 172.21.15.39:0/256807860 shutdown_connections 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 --2- 172.21.15.39:0/256807860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb2810dfa0 0x7fbb28110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 --2- 172.21.15.39:0/256807860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb28100da0 0x7fbb2810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 --2- 172.21.15.39:0/256807860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb281003f0 0x7fbb281007d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 -- 172.21.15.39:0/256807860 >> 172.21.15.39:0/256807860 conn(0x7fbb280fc030 msgr2=0x7fbb280fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 -- 172.21.15.39:0/256807860 shutdown_connections 2024-03-16T10:25:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.989+0000 7fbb27e00640 1 -- 172.21.15.39:0/256807860 wait complete. 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.990+0000 7fbb27e00640 1 Processor -- start 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.990+0000 7fbb27e00640 1 -- start start 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.990+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.990+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 0x7fbb2819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb2810dfa0 0x7fbb2819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb27e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28113c50 con 0x7fbb2810dfa0 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb27e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28113ad0 con 0x7fbb281003f0 2024-03-16T10:25:52.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb27e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28113dd0 con 0x7fbb28100da0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 0x7fbb2819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36118/0 (socket says 172.21.15.39:36118) 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26a00640 1 -- 172.21.15.39:0/2141189563 learned_addr learned my addr 172.21.15.39:0/2141189563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb27400640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb2810dfa0 0x7fbb2819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 -- 172.21.15.39:0/2141189563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 msgr2=0x7fbb2819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 -- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb2810dfa0 msgr2=0x7fbb2819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb2810dfa0 0x7fbb2819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26000640 1 -- 172.21.15.39:0/2141189563 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2819f1e0 con 0x7fbb28100da0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.991+0000 7fbb26a00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb26000640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 0x7fbb2819a730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9f0 tx=0x7fbb1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400ccc0 con 0x7fbb28100da0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb14010910 con 0x7fbb28100da0 2024-03-16T10:25:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2819f4d0 con 0x7fbb28100da0 2024-03-16T10:25:52.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018960 con 0x7fbb28100da0 2024-03-16T10:25:52.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.992+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281a6d60 con 0x7fbb28100da0 2024-03-16T10:25:52.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.993+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb280684f0 con 0x7fbb28100da0 2024-03-16T10:25:52.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.995+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7fbb14010ac0 con 0x7fbb28100da0 2024-03-16T10:25:52.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.997+0000 7fbb1f400640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 0x7fbb0c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:52.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.997+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7fbb140cf570 con 0x7fbb28100da0 2024-03-16T10:25:52.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.997+0000 7fbb26a00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 0x7fbb0c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:52.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:52.998+0000 7fbb26a00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 0x7fbb0c0a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005f70 tx=0x7fbb180058d0 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:53.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.001+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbb1409b870 con 0x7fbb28100da0 2024-03-16T10:25:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.219+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb28109160 con 0x7fbb28100da0 2024-03-16T10:25:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.220+0000 7fbb1f400640 1 -- 172.21.15.39:0/2141189563 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v117) ==== 81+0+944 (secure 0 0 0) 0x7fbb1409f910 con 0x7fbb28100da0 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1082942412 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3265323877 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2375807750 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3114779354 2024-03-17T09:58:12.056328+0000 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3549110681 2024-03-17T09:58:12.056328+0000 2024-03-16T10:25:53.221 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1618736720 2024-03-17T09:58:12.056328+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/332905917 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/865907722 2024-03-17T09:57:37.234906+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4208944392 2024-03-17T09:57:37.234906+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1082942412 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3969870217 2024-03-17T09:57:37.234906+0000 2024-03-16T10:25:53.222 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3617878948 2024-03-17T09:57:37.234906+0000 2024-03-16T10:25:53.223 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2792410890 2024-03-17T10:09:13.856223+0000 2024-03-16T10:25:53.223 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1974903299 2024-03-17T09:58:12.056328+0000 2024-03-16T10:25:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:listed 16 entries 2024-03-16T10:25:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.222+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 msgr2=0x7fbb0c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.222+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 0x7fbb0c0a4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005f70 tx=0x7fbb180058d0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.222+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 msgr2=0x7fbb2819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.222+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 0x7fbb2819a730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400e9f0 tx=0x7fbb1400eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 shutdown_connections 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb2810dfa0 0x7fbb2819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fbb0c0a29f0 0x7fbb0c0a4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb28100da0 0x7fbb2819a730 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 --2- 172.21.15.39:0/2141189563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb281003f0 0x7fbb2819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 >> 172.21.15.39:0/2141189563 conn(0x7fbb280fc030 msgr2=0x7fbb280fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 shutdown_connections 2024-03-16T10:25:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:53.223+0000 7fbb27e00640 1 -- 172.21.15.39:0/2141189563 wait complete. 2024-03-16T10:25:53.244 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T10:25:53.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:25:53.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /tmp/cephfs 2024-03-16T10:25:53.647 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:25:53.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:25:53.648 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-03-16T10:25:53.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:25:53.748 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-03-16T10:25:53.821 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-03-16T10:25:53.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:25:53.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-03-16T10:25:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:53 smithi105 ceph-mon[36559]: pgmap v524: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:53 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3210775358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:53.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:53 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2141189563' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:25:53.883 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:53 smithi039 ceph-mon[31510]: pgmap v524: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:53.883 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:53 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3210775358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:53.883 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:53 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2141189563' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:25:53.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:25:53.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-03-16T10:25:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:53 smithi194 ceph-mon[36445]: pgmap v524: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-03-16T10:25:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:53 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3210775358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-16T10:25:53.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:53 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2141189563' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:25:54.011 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-16T10:25:54.021 INFO:tasks.cephadm:Teardown begin 2024-03-16T10:25:54.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T10:25:54.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 -- 172.21.15.39:0/2637835617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 msgr2=0x7f033c107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 --2- 172.21.15.39:0/2637835617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c107a30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f032c009930 tx=0x7f032c02bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 -- 172.21.15.39:0/2637835617 shutdown_connections 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 --2- 172.21.15.39:0/2637835617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101f50 0x7f033c10e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 --2- 172.21.15.39:0/2637835617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101590 0x7f033c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 --2- 172.21.15.39:0/2637835617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c107a30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 -- 172.21.15.39:0/2637835617 >> 172.21.15.39:0/2637835617 conn(0x7f033c0fd080 msgr2=0x7f033c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 -- 172.21.15.39:0/2637835617 shutdown_connections 2024-03-16T10:25:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.235+0000 7f033b200640 1 -- 172.21.15.39:0/2637835617 wait complete. 2024-03-16T10:25:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.237+0000 7f033b200640 1 Processor -- start 2024-03-16T10:25:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.237+0000 7f033b200640 1 -- start start 2024-03-16T10:25:54.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.237+0000 7f033b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101590 0x7f033c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033b200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033b200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f033c1119a0 con 0x7f033c101590 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033b200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f033c111820 con 0x7f033c101f50 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033b200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f033c111b20 con 0x7f033c107650 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36128/0 (socket says 172.21.15.39:36128) 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 -- 172.21.15.39:0/1508273885 learned_addr learned my addr 172.21.15.39:0/1508273885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339e00640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101590 0x7f033c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f033a800640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 -- 172.21.15.39:0/1508273885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 msgr2=0x7f033c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 -- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101590 msgr2=0x7f033c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101590 0x7f033c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.238+0000 7f0339400640 1 -- 172.21.15.39:0/1508273885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c19ef70 con 0x7f033c101f50 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f0339400640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f032800ea60 tx=0x7f032800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:54.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800cde0 con 0x7f033c101f50 2024-03-16T10:25:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03280035f0 con 0x7f033c101f50 2024-03-16T10:25:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c19f260 con 0x7f033c101f50 2024-03-16T10:25:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328010670 con 0x7f033c101f50 2024-03-16T10:25:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.239+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c1a6af0 con 0x7f033c101f50 2024-03-16T10:25:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.240+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033c102cb0 con 0x7f033c101f50 2024-03-16T10:25:54.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.242+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f03280037a0 con 0x7f033c101f50 2024-03-16T10:25:54.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.244+0000 7f0332a00640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 0x7f03200a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.244+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f03280cf5c0 con 0x7f033c101f50 2024-03-16T10:25:54.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.244+0000 7f0339e00640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 0x7f03200a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.245+0000 7f0339e00640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 0x7f03200a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f032c009900 tx=0x7f032c02df90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.248+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f032809beb0 con 0x7f033c101f50 2024-03-16T10:25:54.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.565+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f033c0684f0 con 0x7f033c101f50 2024-03-16T10:25:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.566+0000 7f0332a00640 1 -- 172.21.15.39:0/1508273885 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11947 (secure 0 0 0) 0x7f032809b7b0 con 0x7f033c101f50 2024-03-16T10:25:54.567 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:54.567 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:24:52.478246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24557,24518,24544,24523]},"id":1}]} 2024-03-16T10:25:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 msgr2=0x7f03200a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 0x7f03200a4e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f032c009900 tx=0x7f032c02df90 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 msgr2=0x7f033c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f032800ea60 tx=0x7f032800ef20 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 shutdown_connections 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f03200a2970 0x7f03200a4e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f033c107650 0x7f033c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f033c101f50 0x7f033c19a4c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 --2- 172.21.15.39:0/1508273885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f033c101590 0x7f033c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.569+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 >> 172.21.15.39:0/1508273885 conn(0x7f033c0fd080 msgr2=0x7f033c10cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.570+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 shutdown_connections 2024-03-16T10:25:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.570+0000 7f033b200640 1 -- 172.21.15.39:0/1508273885 wait complete. 2024-03-16T10:25:54.593 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:25:54.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 -- 172.21.15.39:0/3285333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c069f40 msgr2=0x7f929c06a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 --2- 172.21.15.39:0/3285333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c069f40 0x7f929c06a3c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9288009960 tx=0x7f928802bc10 comp rx=0 tx=0).stop 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 -- 172.21.15.39:0/3285333482 shutdown_connections 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 --2- 172.21.15.39:0/3285333482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c06a900 0x7f929c10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 --2- 172.21.15.39:0/3285333482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c069f40 0x7f929c06a3c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 --2- 172.21.15.39:0/3285333482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 -- 172.21.15.39:0/3285333482 >> 172.21.15.39:0/3285333482 conn(0x7f929c076a60 msgr2=0x7f929c076e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 -- 172.21.15.39:0/3285333482 shutdown_connections 2024-03-16T10:25:54.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.806+0000 7f92a4200640 1 -- 172.21.15.39:0/3285333482 wait complete. 2024-03-16T10:25:54.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 Processor -- start 2024-03-16T10:25:54.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 -- start start 2024-03-16T10:25:54.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 0x7f929c1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c101120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f929c111be0 con 0x7f929c107840 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f929c111a60 con 0x7f929c06a900 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.807+0000 7f92a4200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f929c111d60 con 0x7f929c069f40 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a1800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 0x7f929c1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a1800640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 0x7f929c1006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56920/0 (socket says 172.21.15.39:56920) 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36138/0 (socket says 172.21.15.39:36138) 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a1800640 1 -- 172.21.15.39:0/1398244789 learned_addr learned my addr 172.21.15.39:0/1398244789 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a2200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 -- 172.21.15.39:0/1398244789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 msgr2=0x7f929c1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 0x7f929c1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 -- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 msgr2=0x7f929c101120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c101120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 -- 172.21.15.39:0/1398244789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c1a6520 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a2200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.808+0000 7f92a0e00640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f928802c150 tx=0x7f92880042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.809+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288043020 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.809+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92880054e0 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.809+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929c1a67b0 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.809+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92880039e0 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.809+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929c1a6d20 con 0x7f929c06a900 2024-03-16T10:25:54.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.810+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929c068a90 con 0x7f929c06a900 2024-03-16T10:25:54.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.812+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f9288046020 con 0x7f929c06a900 2024-03-16T10:25:54.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.813+0000 7f929a000640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 0x7f92840a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:54.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.813+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f92880f0f90 con 0x7f929c06a900 2024-03-16T10:25:54.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.814+0000 7f92a1800640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 0x7f92840a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:54.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.814+0000 7f92a1800640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 0x7f92840a4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9290009c10 tx=0x7f9290005f90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:54.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:54.817+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f928804d020 con 0x7f929c06a900 2024-03-16T10:25:55.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.064+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f929c02d1b0 con 0x7f929c06a900 2024-03-16T10:25:55.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.066+0000 7f929a000640 1 -- 172.21.15.39:0/1398244789 <== mon.2 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11947 (secure 0 0 0) 0x7f92880bd9f0 con 0x7f929c06a900 2024-03-16T10:25:55.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:55.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/428058525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":428058525},{"type":"v1","addr":"172.21.15.39:6839","nonce":428058525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/2269577878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2269577878},{"type":"v1","addr":"172.21.15.105:6833","nonce":2269577878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T10:06:03.734324+0000","modified":"2024-03-16T10:24:52.478246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544,"mds_3":24523,"mds_4":24518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6833/1279561047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1279561047},{"type":"v1","addr":"172.21.15.194:6833","nonce":1279561047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6835/1951962450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1951962450},{"type":"v1","addr":"172.21.15.39:6835","nonce":1951962450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/2605661802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2605661802},{"type":"v1","addr":"172.21.15.39:6837","nonce":2605661802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6835/106855660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":106855660},{"type":"v1","addr":"172.21.15.105:6835","nonce":106855660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/3003167044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":3003167044},{"type":"v1","addr":"172.21.15.194:6835","nonce":3003167044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.105:6837/3830303638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3830303638},{"type":"v1","addr":"172.21.15.105:6837","nonce":3830303638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.194:6837/3973444513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":3973444513},{"type":"v1","addr":"172.21.15.194:6837","nonce":3973444513}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.194:6839/985236448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":985236448},{"type":"v1","addr":"172.21.15.194:6839","nonce":985236448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6841/1425412361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":1425412361},{"type":"v1","addr":"172.21.15.39:6841","nonce":1425412361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6839/2189492991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":2189492991},{"type":"v1","addr":"172.21.15.105:6839","nonce":2189492991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24569,24557,24518,24544,24523]},"id":1}]} 2024-03-16T10:25:55.066 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-03-16T10:25:55.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 msgr2=0x7f92840a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 0x7f92840a4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9290009c10 tx=0x7f9290005f90 comp rx=0 tx=0).stop 2024-03-16T10:25:55.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 msgr2=0x7f929c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f928802c150 tx=0x7f92880042f0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 shutdown_connections 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c107840 0x7f929c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f929c06a900 0x7f929c100be0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f92840a2a20 0x7f92840a4ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 --2- 172.21.15.39:0/1398244789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f929c069f40 0x7f929c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 >> 172.21.15.39:0/1398244789 conn(0x7f929c076a60 msgr2=0x7f929c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.068+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 shutdown_connections 2024-03-16T10:25:55.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.069+0000 7f92a4200640 1 -- 172.21.15.39:0/1398244789 wait complete. 2024-03-16T10:25:55.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 -- 172.21.15.39:0/4009905577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc1037a0 msgr2=0x7f26fc103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 --2- 172.21.15.39:0/4009905577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc1037a0 0x7f26fc103c20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009900 tx=0x7f26e402bc30 comp rx=0 tx=0).stop 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 -- 172.21.15.39:0/4009905577 shutdown_connections 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 --2- 172.21.15.39:0/4009905577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc104160 0x7f26fc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 --2- 172.21.15.39:0/4009905577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc1037a0 0x7f26fc103c20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 --2- 172.21.15.39:0/4009905577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1097b0 0x7f26fc109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.302+0000 7f2701600640 1 -- 172.21.15.39:0/4009905577 >> 172.21.15.39:0/4009905577 conn(0x7f26fc0ff250 msgr2=0x7f26fc101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:55.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.303+0000 7f2701600640 1 -- 172.21.15.39:0/4009905577 shutdown_connections 2024-03-16T10:25:55.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.303+0000 7f2701600640 1 -- 172.21.15.39:0/4009905577 wait complete. 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 Processor -- start 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 -- start start 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 0x7f26fc19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 0x7f26fc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc1097b0 0x7f26fc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc113c10 con 0x7f26fc1097b0 2024-03-16T10:25:55.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc113a90 con 0x7f26fc104160 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2701600640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc113d90 con 0x7f26fc1037a0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f26f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 0x7f26fc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f26fbe00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 0x7f26fc19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f26f3e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 0x7f26fc19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.39:36150/0 (socket says 172.21.15.39:36150) 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f26f3e00640 1 -- 172.21.15.39:0/2510595383 learned_addr learned my addr 172.21.15.39:0/2510595383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.304+0000 7f2700c00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc1097b0 0x7f26fc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 -- 172.21.15.39:0/2510595383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 msgr2=0x7f26fc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 0x7f26fc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 -- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc1097b0 msgr2=0x7f26fc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc1097b0 0x7f26fc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 -- 172.21.15.39:0/2510595383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc19f020 con 0x7f26fc1037a0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26fbe00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 0x7f26fc19a030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9f0 tx=0x7f26ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00cd80 con 0x7f26fc1037a0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc19f310 con 0x7f26fc1037a0 2024-03-16T10:25:55.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec0035f0 con 0x7f26fc1037a0 2024-03-16T10:25:55.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.305+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec010670 con 0x7f26fc1037a0 2024-03-16T10:25:55.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.306+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a6ba0 con 0x7f26fc1037a0 2024-03-16T10:25:55.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.307+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26fc068940 con 0x7f26fc1037a0 2024-03-16T10:25:55.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.309+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f26ec0037a0 con 0x7f26fc1037a0 2024-03-16T10:25:55.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.310+0000 7f26f9600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 0x7f26e00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.310+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7f26ec0cf500 con 0x7f26fc1037a0 2024-03-16T10:25:55.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.310+0000 7f26f3e00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 0x7f26e00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.311+0000 7f26f3e00640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 0x7f26e00a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26fc101800 tx=0x7f26e4005a20 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:55.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.314+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26ec09bdf0 con 0x7f26fc1037a0 2024-03-16T10:25:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.541+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26fc10bd70 con 0x7f26fc1037a0 2024-03-16T10:25:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.543+0000 7f26f9600640 1 -- 172.21.15.39:0/2510595383 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+25415 (secure 0 0 0) 0x7f26ec09b6f0 con 0x7f26fc1037a0 2024-03-16T10:25:55.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-16T10:25:55.544 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"fsid":"65a28618-e37b-11ee-95c9-87774f69a715","created":"2024-03-16T09:57:03.274650+0000","modified":"2024-03-16T10:25:12.293141+0000","last_up_change":"2024-03-16T10:05:07.830036+0000","last_in_change":"2024-03-16T10:04:48.180503+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:00:56.748725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:06:01.836087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:06:03.184357+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-16T10:06:06.832116+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f36f2e74-95e3-4d82-a0f2-a19bf0b67dc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6803","nonce":1765190956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6805","nonce":1765190956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6809","nonce":1765190956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1765190956},{"type":"v1","addr":"172.21.15.39:6807","nonce":1765190956}]},"public_addr":"172.21.15.39:6803/1765190956","cluster_addr":"172.21.15.39:6805/1765190956","heartbeat_back_addr":"172.21.15.39:6809/1765190956","heartbeat_front_addr":"172.21.15.39:6807/1765190956","state":["exists","up"]},{"osd":1,"uuid":"097d0219-dbb6-485b-a385-33b389bf02c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6801","nonce":3759059886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6803","nonce":3759059886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6807","nonce":3759059886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3759059886},{"type":"v1","addr":"172.21.15.105:6805","nonce":3759059886}]},"public_addr":"172.21.15.105:6801/3759059886","cluster_addr":"172.21.15.105:6803/3759059886","heartbeat_back_addr":"172.21.15.105:6807/3759059886","heartbeat_front_addr":"172.21.15.105:6805/3759059886","state":["exists","up"]},{"osd":2,"uuid":"1f3873f3-0b87-42c9-a34b-f62099b77f39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6801","nonce":3029390216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6803","nonce":3029390216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6807","nonce":3029390216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3029390216},{"type":"v1","addr":"172.21.15.194:6805","nonce":3029390216}]},"public_addr":"172.21.15.194:6801/3029390216","cluster_addr":"172.21.15.194:6803/3029390216","heartbeat_back_addr":"172.21.15.194:6807/3029390216","heartbeat_front_addr":"172.21.15.194:6805/3029390216","state":["exists","up"]},{"osd":3,"uuid":"a6455fd4-f277-4db1-a5c8-0fb262d3bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6811","nonce":2181780282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6813","nonce":2181780282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6817","nonce":2181780282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2181780282},{"type":"v1","addr":"172.21.15.39:6815","nonce":2181780282}]},"public_addr":"172.21.15.39:6811/2181780282","cluster_addr":"172.21.15.39:6813/2181780282","heartbeat_back_addr":"172.21.15.39:6817/2181780282","heartbeat_front_addr":"172.21.15.39:6815/2181780282","state":["exists","up"]},{"osd":4,"uuid":"3d166e33-4a4b-48ce-b293-7d6fe30b2adb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6809","nonce":45639064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6811","nonce":45639064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6815","nonce":45639064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":45639064},{"type":"v1","addr":"172.21.15.105:6813","nonce":45639064}]},"public_addr":"172.21.15.105:6809/45639064","cluster_addr":"172.21.15.105:6811/45639064","heartbeat_back_addr":"172.21.15.105:6815/45639064","heartbeat_front_addr":"172.21.15.105:6813/45639064","state":["exists","up"]},{"osd":5,"uuid":"2d6fad5d-00d3-4e3b-b28f-8e1a9f037c00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6809","nonce":846499342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6811","nonce":846499342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6815","nonce":846499342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":846499342},{"type":"v1","addr":"172.21.15.194:6813","nonce":846499342}]},"public_addr":"172.21.15.194:6809/846499342","cluster_addr":"172.21.15.194:6811/846499342","heartbeat_back_addr":"172.21.15.194:6815/846499342","heartbeat_front_addr":"172.21.15.194:6813/846499342","state":["exists","up"]},{"osd":6,"uuid":"f4d56835-6eb8-4763-bc1e-da513d084bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6819","nonce":2687294555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6821","nonce":2687294555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6825","nonce":2687294555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":2687294555},{"type":"v1","addr":"172.21.15.39:6823","nonce":2687294555}]},"public_addr":"172.21.15.39:6819/2687294555","cluster_addr":"172.21.15.39:6821/2687294555","heartbeat_back_addr":"172.21.15.39:6825/2687294555","heartbeat_front_addr":"172.21.15.39:6823/2687294555","state":["exists","up"]},{"osd":7,"uuid":"14b5763b-c967-49ad-a8d6-e37f49999f9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6817","nonce":2696110582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6819","nonce":2696110582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6823","nonce":2696110582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2696110582},{"type":"v1","addr":"172.21.15.105:6821","nonce":2696110582}]},"public_addr":"172.21.15.105:6817/2696110582","cluster_addr":"172.21.15.105:6819/2696110582","heartbeat_back_addr":"172.21.15.105:6823/2696110582","heartbeat_front_addr":"172.21.15.105:6821/2696110582","state":["exists","up"]},{"osd":8,"uuid":"9e0fa1af-48e7-4764-ad45-390850e2bc7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6817","nonce":4066786561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6819","nonce":4066786561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6823","nonce":4066786561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":4066786561},{"type":"v1","addr":"172.21.15.194:6821","nonce":4066786561}]},"public_addr":"172.21.15.194:6817/4066786561","cluster_addr":"172.21.15.194:6819/4066786561","heartbeat_back_addr":"172.21.15.194:6823/4066786561","heartbeat_front_addr":"172.21.15.194:6821/4066786561","state":["exists","up"]},{"osd":9,"uuid":"c112bf93-06c2-409e-8f31-837fd3d2b600","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6827","nonce":2744339033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6829","nonce":2744339033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6833","nonce":2744339033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2744339033},{"type":"v1","addr":"172.21.15.39:6831","nonce":2744339033}]},"public_addr":"172.21.15.39:6827/2744339033","cluster_addr":"172.21.15.39:6829/2744339033","heartbeat_back_addr":"172.21.15.39:6833/2744339033","heartbeat_front_addr":"172.21.15.39:6831/2744339033","state":["exists","up"]},{"osd":10,"uuid":"f44ef9e2-7391-4298-a185-199839c37867","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6825","nonce":4246880500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6827","nonce":4246880500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6831","nonce":4246880500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":4246880500},{"type":"v1","addr":"172.21.15.105:6829","nonce":4246880500}]},"public_addr":"172.21.15.105:6825/4246880500","cluster_addr":"172.21.15.105:6827/4246880500","heartbeat_back_addr":"172.21.15.105:6831/4246880500","heartbeat_front_addr":"172.21.15.105:6829/4246880500","state":["exists","up"]},{"osd":11,"uuid":"9a171911-8bf9-4ad4-ab72-98d5c48d0807","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6825","nonce":254372384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6827","nonce":254372384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6831","nonce":254372384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":254372384},{"type":"v1","addr":"172.21.15.194:6829","nonce":254372384}]},"public_addr":"172.21.15.194:6825/254372384","cluster_addr":"172.21.15.194:6827/254372384","heartbeat_back_addr":"172.21.15.194:6831/254372384","heartbeat_front_addr":"172.21.15.194:6829/254372384","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:05.454871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:29.680062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:00:52.922798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:25.242888+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:01:51.301004+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:15.623385+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:02:49.248893+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:15.155819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:03:39.930084+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:13.500981+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:04:40.369048+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:05:06.333520+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.11","mappings":[{"from":5,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3265323877":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/2375807750":"2024-03-17T10:09:13.856223+0000","172.21.15.39:0/3114779354":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/3549110681":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/1618736720":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/332905917":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/1974903299":"2024-03-17T09:58:12.056328+0000","172.21.15.39:0/865907722":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/4208944392":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6800/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:6801/1082942412":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6801/3969870217":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/3617878948":"2024-03-17T09:57:37.234906+0000","172.21.15.39:0/2792410890":"2024-03-17T10:09:13.856223+0000","172.21.15.39:6800/1974903299":"2024-03-17T09:58:12.056328+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 msgr2=0x7f26e00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 0x7f26e00a4e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26fc101800 tx=0x7f26e4005a20 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 msgr2=0x7f26fc19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 0x7f26fc19a030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00e9f0 tx=0x7f26ec00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 shutdown_connections 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26fc1097b0 0x7f26fc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26fc104160 0x7f26fc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f26e00a2970 0x7f26e00a4e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 --2- 172.21.15.39:0/2510595383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26fc1037a0 0x7f26fc19a030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 >> 172.21.15.39:0/2510595383 conn(0x7f26fc0ff250 msgr2=0x7f26fc0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 shutdown_connections 2024-03-16T10:25:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.546+0000 7f2701600640 1 -- 172.21.15.39:0/2510595383 wait complete. 2024-03-16T10:25:55.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 -- 172.21.15.39:0/2536751390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb174073bb0 msgr2=0x7fb174073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 --2- 172.21.15.39:0/2536751390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb174073bb0 0x7fb174073f90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb164009900 tx=0x7fb16402bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 -- 172.21.15.39:0/2536751390 shutdown_connections 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 --2- 172.21.15.39:0/2536751390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb17410def0 0x7fb174110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 --2- 172.21.15.39:0/2536751390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1740744d0 0x7fb17410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 --2- 172.21.15.39:0/2536751390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb174073bb0 0x7fb174073f90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.777+0000 7fb178e00640 1 -- 172.21.15.39:0/2536751390 >> 172.21.15.39:0/2536751390 conn(0x7fb1740fd180 msgr2=0x7fb1740ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:55.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.778+0000 7fb178e00640 1 -- 172.21.15.39:0/2536751390 shutdown_connections 2024-03-16T10:25:55.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.778+0000 7fb178e00640 1 -- 172.21.15.39:0/2536751390 wait complete. 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 Processor -- start 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 -- start start 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb174073bb0 0x7fb17419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 0x7fb17419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113cc0 con 0x7fb174073bb0 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113b40 con 0x7fb17410def0 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb178e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb174113e40 con 0x7fb1740744d0 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb172a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb173e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 0x7fb17419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb172a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:56950/0 (socket says 172.21.15.39:56950) 2024-03-16T10:25:55.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb172a00640 1 -- 172.21.15.39:0/172681598 learned_addr learned my addr 172.21.15.39:0/172681598 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.779+0000 7fb173400640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb174073bb0 0x7fb17419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 -- 172.21.15.39:0/172681598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 msgr2=0x7fb17419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 0x7fb17419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 -- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb174073bb0 msgr2=0x7fb17419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb174073bb0 0x7fb17419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 -- 172.21.15.39:0/172681598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17419f170 con 0x7fb1740744d0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb173e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 0x7fb17419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb172a00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9f0 tx=0x7fb15c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cd80 con 0x7fb1740744d0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c010910 con 0x7fb1740744d0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17419f460 con 0x7fb1740744d0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.780+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c018990 con 0x7fb1740744d0 2024-03-16T10:25:55.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.781+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741a6cf0 con 0x7fb1740744d0 2024-03-16T10:25:55.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.782+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1740684f0 con 0x7fb1740744d0 2024-03-16T10:25:55.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.783+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7fb15c010ac0 con 0x7fb1740744d0 2024-03-16T10:25:55.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.785+0000 7fb16be00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 0x7fb1580a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:55.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.785+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9562+0+0 (secure 0 0 0) 0x7fb15c0cf570 con 0x7fb1740744d0 2024-03-16T10:25:55.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.785+0000 7fb173400640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 0x7fb1580a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.786+0000 7fb173400640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 0x7fb1580a4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb164005ab0 tx=0x7fb164005a20 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:55.793+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb15c09b810 con 0x7fb1740744d0 2024-03-16T10:25:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:55 smithi105 ceph-mon[36559]: pgmap v525: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1508273885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/1398244789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:55.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:55 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2510595383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:55.955 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:55 smithi194 ceph-mon[36445]: pgmap v525: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1508273885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/1398244789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:55.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:55 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2510595383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:55 smithi039 ceph-mon[31510]: pgmap v525: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-03-16T10:25:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1508273885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/1398244789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:25:56.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:55 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2510595383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:25:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.041+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1741090a0 con 0x7fb1740744d0 2024-03-16T10:25:56.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.051+0000 7fb16be00640 1 -- 172.21.15.39:0/172681598 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7fb15c09f8b0 con 0x7fb1740744d0 2024-03-16T10:25:56.051 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T10:25:56.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.055+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 msgr2=0x7fb1580a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:56.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.055+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 0x7fb1580a4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb164005ab0 tx=0x7fb164005a20 comp rx=0 tx=0).stop 2024-03-16T10:25:56.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.055+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 msgr2=0x7fb17419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:56.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.055+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9f0 tx=0x7fb15c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 shutdown_connections 2024-03-16T10:25:56.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb17410def0 0x7fb17419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fb1580a29f0 0x7fb1580a4eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1740744d0 0x7fb17419a6c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 --2- 172.21.15.39:0/172681598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb174073bb0 0x7fb17419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 >> 172.21.15.39:0/172681598 conn(0x7fb1740fd180 msgr2=0x7fb1740fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:56.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 shutdown_connections 2024-03-16T10:25:56.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.056+0000 7fb178e00640 1 -- 172.21.15.39:0/172681598 wait complete. 2024-03-16T10:25:56.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T10:25:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 -- 172.21.15.39:0/1330867986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 msgr2=0x7f8574107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 --2- 172.21.15.39:0/1330867986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f8574107a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8564009900 tx=0x7f856402bc40 comp rx=0 tx=0).stop 2024-03-16T10:25:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 -- 172.21.15.39:0/1330867986 shutdown_connections 2024-03-16T10:25:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 --2- 172.21.15.39:0/1330867986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 0x7f857410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 --2- 172.21.15.39:0/1330867986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f8574101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 --2- 172.21.15.39:0/1330867986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f8574107a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 -- 172.21.15.39:0/1330867986 >> 172.21.15.39:0/1330867986 conn(0x7f85740fd150 msgr2=0x7f85740ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:56.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 -- 172.21.15.39:0/1330867986 shutdown_connections 2024-03-16T10:25:56.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.301+0000 7f8579e00640 1 -- 172.21.15.39:0/1330867986 wait complete. 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 Processor -- start 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 -- start start 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 0x7f857419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f857419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111a70 con 0x7f8574102000 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85741118f0 con 0x7f8574107650 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.302+0000 7f8579e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8574111bf0 con 0x7f8574101640 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:57018/0 (socket says 172.21.15.39:57018) 2024-03-16T10:25:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 -- 172.21.15.39:0/420870349 learned_addr learned my addr 172.21.15.39:0/420870349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8573e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 0x7f857419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8579400640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f857419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 -- 172.21.15.39:0/420870349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 msgr2=0x7f857419e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f857419e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 -- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 msgr2=0x7f857419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 0x7f857419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 -- 172.21.15.39:0/420870349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857419f100 con 0x7f8574101640 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.303+0000 7f8578a00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8564005ab0 tx=0x7f8564004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.304+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564041070 con 0x7f8574101640 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.304+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857419f390 con 0x7f8574101640 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.304+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856402ccd0 con 0x7f8574101640 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.304+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564003e20 con 0x7f8574101640 2024-03-16T10:25:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.304+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85741a6c80 con 0x7f8574101640 2024-03-16T10:25:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.305+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85740684f0 con 0x7f8574101640 2024-03-16T10:25:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.307+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f8564044070 con 0x7f8574101640 2024-03-16T10:25:56.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.308+0000 7f8571600640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 0x7f85580a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:56.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.308+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9904+0+0 (secure 0 0 0) 0x7f85640f0d90 con 0x7f8574101640 2024-03-16T10:25:56.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.308+0000 7f8573e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 0x7f85580a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.309+0000 7f8573e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 0x7f85580a4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85740ff590 tx=0x7f856000b390 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:56.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.312+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85640bd010 con 0x7f8574101640 2024-03-16T10:25:56.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a[31506]: 2024-03-16T10:25:56.043+0000 7f4931a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:25:56.505 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.505 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mds[73582]: -- 172.21.15.39:0/584507418 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564de7f85a40 con 0x564de7f58800 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: starting mds.a at 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mds[72248]: -- 172.21.15.39:0/3996050068 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f60ce1a40 con 0x560f60cb5000 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: starting mds.j at 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.506 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.507 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mds[72598]: -- 172.21.15.39:0/540587369 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575c98afa40 con 0x5575c9883000 2024-03-16T10:25:56.507 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: starting mds.d at 2024-03-16T10:25:56.566 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: starting mds.e at 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mds[67218]: -- 172.21.15.105:0/2527737257 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb37005a40 con 0x55bb36fd8c00 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: starting mds.b at 2024-03-16T10:25:56.567 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h[66902]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.568 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h[66902]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.568 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mds[66906]: -- 172.21.15.105:0/830908845 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618e0b6da40 con 0x5618e0b40800 2024-03-16T10:25:56.568 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h[66902]: starting mds.h at 2024-03-16T10:25:56.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:56.569+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f85741a6e50 con 0x7f8574101640 2024-03-16T10:25:56.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mds[67680]: -- 172.21.15.194:0/1577018164 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55795128da40 con 0x557951260800 2024-03-16T10:25:56.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: starting mds.c at 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mds[67987]: -- 172.21.15.194:0/823230195 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575363e7a40 con 0x5575363ba800 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: starting mds.i at 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mds[67067]: -- 172.21.15.194:0/2238426987 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c05da97a40 con 0x55c05da6a800 2024-03-16T10:25:56.707 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: starting mds.l at 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/172681598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: osdmap e118: 12 total, 12 up, 12 in 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mon[36445]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f[67372]: ignoring --setuser ceph since I am not root 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f[67372]: ignoring --setgroup ceph since I am not root 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-mds[67376]: -- 172.21.15.194:0/2939404093 <== mon.2 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56406c9dba40 con 0x56406c9ae800 2024-03-16T10:25:56.708 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:25:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f[67372]: starting mds.f at 2024-03-16T10:25:56.864 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/172681598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: osdmap e118: 12 total, 12 up, 12 in 2024-03-16T10:25:56.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:56 smithi105 ceph-mon[36559]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-16T10:25:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/172681598' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T10:25:57.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:25:57.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:25:57.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T10:25:57.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: osdmap e118: 12 total, 12 up, 12 in 2024-03-16T10:25:57.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:56 smithi039 ceph-mon[31510]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-16T10:25:57.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.060+0000 7f8571600640 1 -- 172.21.15.39:0/420870349 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f85640c10b0 con 0x7f8574101640 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.064+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 msgr2=0x7f85580a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.064+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 0x7f85580a4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85740ff590 tx=0x7f856000b390 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.064+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 msgr2=0x7f857419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.064+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8564005ab0 tx=0x7f8564004430 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 shutdown_connections 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8574107650 0x7f857419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8574102000 0x7f857419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f85580a29f0 0x7f85580a4eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 --2- 172.21.15.39:0/420870349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8574101640 0x7f857419a1b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 >> 172.21.15.39:0/420870349 conn(0x7f85740fd150 msgr2=0x7f857410c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:57.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.065+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 shutdown_connections 2024-03-16T10:25:57.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.066+0000 7f8579e00640 1 -- 172.21.15.39:0/420870349 wait complete. 2024-03-16T10:25:57.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 -- 172.21.15.39:0/1559978155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a8073b90 msgr2=0x7fc9a8073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 --2- 172.21.15.39:0/1559978155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a8073b90 0x7fc9a8073f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc998009900 tx=0x7fc99802bcc0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 -- 172.21.15.39:0/1559978155 shutdown_connections 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 --2- 172.21.15.39:0/1559978155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a810df10 0x7fc9a8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 --2- 172.21.15.39:0/1559978155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a80744b0 0x7fc9a810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 --2- 172.21.15.39:0/1559978155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a8073b90 0x7fc9a8073f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.305+0000 7fc9a7200640 1 -- 172.21.15.39:0/1559978155 >> 172.21.15.39:0/1559978155 conn(0x7fc9a80fd140 msgr2=0x7fc9a80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:57.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.306+0000 7fc9a7200640 1 -- 172.21.15.39:0/1559978155 shutdown_connections 2024-03-16T10:25:57.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.306+0000 7fc9a7200640 1 -- 172.21.15.39:0/1559978155 wait complete. 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.306+0000 7fc9a7200640 1 Processor -- start 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 -- start start 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a8073b90 0x7fc9a819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 0x7fc9a819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8113c30 con 0x7fc9a810df10 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8113ab0 con 0x7fc9a80744b0 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a7200640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8113db0 con 0x7fc9a8073b90 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a5e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a8073b90 0x7fc9a819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:57.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51798/0 (socket says 172.21.15.39:51798) 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a6800640 1 -- 172.21.15.39:0/2441154681 learned_addr learned my addr 172.21.15.39:0/2441154681 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.307+0000 7fc9a5400640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 0x7fc9a819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 -- 172.21.15.39:0/2441154681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a8073b90 msgr2=0x7fc9a819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a8073b90 0x7fc9a819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 -- 172.21.15.39:0/2441154681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 msgr2=0x7fc9a819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 0x7fc9a819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 -- 172.21.15.39:0/2441154681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a819f130 con 0x7fc9a810df10 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a5400640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 0x7fc9a819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc9a6800640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000eb10 tx=0x7fc9a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.308+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0004b20 con 0x7fc9a810df10 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.309+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a0004cc0 con 0x7fc9a810df10 2024-03-16T10:25:57.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.309+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a819f420 con 0x7fc9a810df10 2024-03-16T10:25:57.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.309+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0010760 con 0x7fc9a810df10 2024-03-16T10:25:57.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.309+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a81a6cb0 con 0x7fc9a810df10 2024-03-16T10:25:57.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.310+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a8068940 con 0x7fc9a810df10 2024-03-16T10:25:57.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.312+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7fc9a0005020 con 0x7fc9a810df10 2024-03-16T10:25:57.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.313+0000 7fc99ea00640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 0x7fc98c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:57.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.314+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9904+0+0 (secure 0 0 0) 0x7fc9a00cf8d0 con 0x7fc9a810df10 2024-03-16T10:25:57.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.314+0000 7fc9a5e00640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 0x7fc98c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:57.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.315+0000 7fc9a5e00640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 0x7fc98c0a4e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc99802df90 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:57.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.319+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc9a009c0d0 con 0x7fc9a810df10 2024-03-16T10:25:57.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:57.525+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc9a81090c0 con 0x7fc9a810df10 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: pgmap v527: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/420870349' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:25:57.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: Cluster is now healthy 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:25:57.866 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6834/404190708,v1:172.21.15.105:6835/404190708] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6836/444202902,v1:172.21.15.105:6838/444202902] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.105:6837/138892895,v1:172.21.15.105:6839/138892895] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6834/2320158032,v1:172.21.15.39:6836/2320158032] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6835/3375322673,v1:172.21.15.39:6837/3375322673] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.39:6840/153245671,v1:172.21.15.39:6841/153245671] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6834/608736384,v1:172.21.15.194:6835/608736384] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6836/906230566,v1:172.21.15.194:6838/906230566] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6837/2464279776,v1:172.21.15.194:6839/2464279776] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: mds.? [v2:172.21.15.194:6832/374069978,v1:172.21.15.194:6833/374069978] up:boot 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: fsmap 12 up:standby 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:57.867 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:57 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: pgmap v527: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-16T10:25:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/420870349' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:25:57.956 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: Cluster is now healthy 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6834/404190708,v1:172.21.15.105:6835/404190708] up:boot 2024-03-16T10:25:57.957 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6836/444202902,v1:172.21.15.105:6838/444202902] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.105:6837/138892895,v1:172.21.15.105:6839/138892895] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6834/2320158032,v1:172.21.15.39:6836/2320158032] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6835/3375322673,v1:172.21.15.39:6837/3375322673] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.39:6840/153245671,v1:172.21.15.39:6841/153245671] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6834/608736384,v1:172.21.15.194:6835/608736384] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6836/906230566,v1:172.21.15.194:6838/906230566] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6837/2464279776,v1:172.21.15.194:6839/2464279776] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: mds.? [v2:172.21.15.194:6832/374069978,v1:172.21.15.194:6833/374069978] up:boot 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: fsmap 12 up:standby 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:57.958 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:57 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: pgmap v527: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/420870349' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: Cluster is now healthy 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T10:25:58.007 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6834/404190708,v1:172.21.15.105:6835/404190708] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6836/444202902,v1:172.21.15.105:6838/444202902] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.105:6837/138892895,v1:172.21.15.105:6839/138892895] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6834/2320158032,v1:172.21.15.39:6836/2320158032] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6835/3375322673,v1:172.21.15.39:6837/3375322673] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.39:6840/153245671,v1:172.21.15.39:6841/153245671] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6834/608736384,v1:172.21.15.194:6835/608736384] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6836/906230566,v1:172.21.15.194:6838/906230566] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6837/2464279776,v1:172.21.15.194:6839/2464279776] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: mds.? [v2:172.21.15.194:6832/374069978,v1:172.21.15.194:6833/374069978] up:boot 2024-03-16T10:25:58.008 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: fsmap 12 up:standby 2024-03-16T10:25:58.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='mgr.25460 172.21.15.39:0/4192953541' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T10:25:58.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:57 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:58.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.060+0000 7fc99ea00640 1 -- 172.21.15.39:0/2441154681 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) ==== 181+0+0 (secure 0 0 0) 0x7fc9a009b9d0 con 0x7fc9a810df10 2024-03-16T10:25:58.061 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-03-16T10:25:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 msgr2=0x7fc98c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 0x7fc98c0a4e30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc99802df90 comp rx=0 tx=0).stop 2024-03-16T10:25:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 msgr2=0x7fc9a819ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:58.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000eb10 tx=0x7fc9a000c2d0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 shutdown_connections 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810df10 0x7fc9a819ea00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9a80744b0 0x7fc9a819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7fc98c0a2970 0x7fc98c0a4e30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 --2- 172.21.15.39:0/2441154681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9a8073b90 0x7fc9a819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 >> 172.21.15.39:0/2441154681 conn(0x7fc9a80fd140 msgr2=0x7fc9a80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 shutdown_connections 2024-03-16T10:25:58.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.065+0000 7fc9a7200640 1 -- 172.21.15.39:0/2441154681 wait complete. 2024-03-16T10:25:58.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T10:25:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 -- 172.21.15.39:0/2149147382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 msgr2=0x7f789410e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 --2- 172.21.15.39:0/2149147382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789410e0e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f788c009900 tx=0x7f788c02bcd0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 -- 172.21.15.39:0/2149147382 shutdown_connections 2024-03-16T10:25:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 --2- 172.21.15.39:0/2149147382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789410e0e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 --2- 172.21.15.39:0/2149147382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894103140 0x7f789410b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 --2- 172.21.15.39:0/2149147382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894102820 0x7f7894102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.325+0000 7f7893e00640 1 -- 172.21.15.39:0/2149147382 >> 172.21.15.39:0/2149147382 conn(0x7f78940fc020 msgr2=0x7f78940fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.326+0000 7f7893e00640 1 -- 172.21.15.39:0/2149147382 shutdown_connections 2024-03-16T10:25:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.326+0000 7f7893e00640 1 -- 172.21.15.39:0/2149147382 wait complete. 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 Processor -- start 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 -- start start 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894102820 0x7f789419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7894111a90 con 0x7f7894103140 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7894111910 con 0x7f789410bc40 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7893e00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7894111c10 con 0x7f7894102820 2024-03-16T10:25:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7892000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7892000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51820/0 (socket says 172.21.15.39:51820) 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.327+0000 7f7892000640 1 -- 172.21.15.39:0/3193501938 learned_addr learned my addr 172.21.15.39:0/3193501938 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892a00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894102820 0x7f789419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7893400640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 -- 172.21.15.39:0/3193501938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894102820 msgr2=0x7f789419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894102820 0x7f789419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 -- 172.21.15.39:0/3193501938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 msgr2=0x7f789419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 -- 172.21.15.39:0/3193501938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789419f160 con 0x7f7894103140 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.328+0000 7f7892000640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.329+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00ccc0 con 0x7f7894103140 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.329+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789419f450 con 0x7f7894103140 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.329+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787c010910 con 0x7f7894103140 2024-03-16T10:25:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.329+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78941a6ce0 con 0x7f7894103140 2024-03-16T10:25:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.329+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c0188c0 con 0x7f7894103140 2024-03-16T10:25:58.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.330+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7894068940 con 0x7f7894103140 2024-03-16T10:25:58.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.332+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f787c010430 con 0x7f7894103140 2024-03-16T10:25:58.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.333+0000 7f788b400640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 0x7f78780a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.333+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9379+0+0 (secure 0 0 0) 0x7f787c0cf340 con 0x7f7894103140 2024-03-16T10:25:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.334+0000 7f7892a00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 0x7f78780a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:58.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.334+0000 7f7892a00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 0x7f78780a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7884005fd0 tx=0x7f7884009500 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:58.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.337+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f787c09b6f0 con 0x7f7894103140 2024-03-16T10:25:58.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:58.547+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f78941a6f40 con 0x7f7894103140 2024-03-16T10:25:59.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.072+0000 7f788b400640 1 -- 172.21.15.39:0/3193501938 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) ==== 169+0+0 (secure 0 0 0) 0x7f787c09f790 con 0x7f7894103140 2024-03-16T10:25:59.072 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-03-16T10:25:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 msgr2=0x7f78780a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 0x7f78780a4e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7884005fd0 tx=0x7f7884009500 comp rx=0 tx=0).stop 2024-03-16T10:25:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 msgr2=0x7f789419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9f0 tx=0x7f787c00eeb0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 shutdown_connections 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f789410bc40 0x7f789419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7894103140 0x7f789419a6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f78780a2970 0x7f78780a4e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 --2- 172.21.15.39:0/3193501938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7894102820 0x7f789419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.076+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 >> 172.21.15.39:0/3193501938 conn(0x7f78940fc020 msgr2=0x7f78940fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:59.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.077+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 shutdown_connections 2024-03-16T10:25:59.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.077+0000 7f7893e00640 1 -- 172.21.15.39:0/3193501938 wait complete. 2024-03-16T10:25:59.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-16T10:25:59.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 -- 172.21.15.39:0/154617751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec103130 msgr2=0x7f64ec10b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 --2- 172.21.15.39:0/154617751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec103130 0x7f64ec10b610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802bcc0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 -- 172.21.15.39:0/154617751 shutdown_connections 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 --2- 172.21.15.39:0/154617751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec10bc30 0x7f64ec10e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 --2- 172.21.15.39:0/154617751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec103130 0x7f64ec10b610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 --2- 172.21.15.39:0/154617751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 -- 172.21.15.39:0/154617751 >> 172.21.15.39:0/154617751 conn(0x7f64ec0fc030 msgr2=0x7f64ec0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 -- 172.21.15.39:0/154617751 shutdown_connections 2024-03-16T10:25:59.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.313+0000 7f64f2a00640 1 -- 172.21.15.39:0/154617751 wait complete. 2024-03-16T10:25:59.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.314+0000 7f64f2a00640 1 Processor -- start 2024-03-16T10:25:59.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.314+0000 7f64f2a00640 1 -- start start 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.314+0000 7f64f2a00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.314+0000 7f64f2a00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec103130 0x7f64ec19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec10bc30 0x7f64ec19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f2a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec111a80 con 0x7f64ec10bc30 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f2a00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec111900 con 0x7f64ec103130 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f2a00640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec111c00 con 0x7f64ec102810 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.39:57070/0 (socket says 172.21.15.39:57070) 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 -- 172.21.15.39:0/3829821939 learned_addr learned my addr 172.21.15.39:0/3829821939 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec10bc30 0x7f64ec19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f0c00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec103130 0x7f64ec19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 -- 172.21.15.39:0/3829821939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec103130 msgr2=0x7f64ec19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:59.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec103130 0x7f64ec19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 -- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec10bc30 msgr2=0x7f64ec19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec10bc30 0x7f64ec19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 -- 172.21.15.39:0/3829821939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec19f190 con 0x7f64ec102810 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.315+0000 7f64f1600640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64e000e9f0 tx=0x7f64e000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.316+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e000cd80 con 0x7f64ec102810 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.316+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0005500 con 0x7f64ec102810 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.316+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec19f480 con 0x7f64ec102810 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.316+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0010690 con 0x7f64ec102810 2024-03-16T10:25:59.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.316+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1a6d10 con 0x7f64ec102810 2024-03-16T10:25:59.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.317+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec0684f0 con 0x7f64ec102810 2024-03-16T10:25:59.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.319+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 27) ==== 137421+0+0 (secure 0 0 0) 0x7f64e00046c0 con 0x7f64ec102810 2024-03-16T10:25:59.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.320+0000 7f64ea000640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 0x7f64d40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:25:59.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.320+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8754+0+0 (secure 0 0 0) 0x7f64e00cf320 con 0x7f64ec102810 2024-03-16T10:25:59.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.321+0000 7f64f0c00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 0x7f64d40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:25:59.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.321+0000 7f64f0c00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 0x7f64d40a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005ab0 tx=0x7f64d8005a20 comp rx=0 tx=0).ready entity=mgr.25460 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:25:59.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.325+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f64e009bf30 con 0x7f64ec102810 2024-03-16T10:25:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:59 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:25:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:59 smithi105 ceph-mon[36559]: osdmap e119: 12 total, 12 up, 12 in 2024-03-16T10:25:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:59 smithi105 ceph-mon[36559]: pgmap v529: 97 pgs: 97 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:25:59.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:25:59 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:59.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:59 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:25:59.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:59 smithi194 ceph-mon[36445]: osdmap e119: 12 total, 12 up, 12 in 2024-03-16T10:25:59.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:59 smithi194 ceph-mon[36445]: pgmap v529: 97 pgs: 97 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:25:59.457 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:25:59 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:59 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/2441154681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:25:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:59 smithi039 ceph-mon[31510]: osdmap e119: 12 total, 12 up, 12 in 2024-03-16T10:25:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:59 smithi039 ceph-mon[31510]: pgmap v529: 97 pgs: 97 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T10:25:59.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:25:59 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:25:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:25:59.550+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f64ec1a6ee0 con 0x7f64ec102810 2024-03-16T10:26:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.081+0000 7f64ea000640 1 -- 172.21.15.39:0/3829821939 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v121) ==== 178+0+0 (secure 0 0 0) 0x7f64e009b830 con 0x7f64ec102810 2024-03-16T10:26:00.083 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data_ec' removed 2024-03-16T10:26:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 msgr2=0x7f64d40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:26:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 0x7f64d40a4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005ab0 tx=0x7f64d8005a20 comp rx=0 tx=0).stop 2024-03-16T10:26:00.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 msgr2=0x7f64ec19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64e000e9f0 tx=0x7f64e000edf0 comp rx=0 tx=0).stop 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 shutdown_connections 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64ec10bc30 0x7f64ec19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64ec103130 0x7f64ec19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.39:6800/2004847378,v1:172.21.15.39:6801/2004847378] conn(0x7f64d40a29f0 0x7f64d40a4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 --2- 172.21.15.39:0/3829821939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64ec102810 0x7f64ec19a1a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 >> 172.21.15.39:0/3829821939 conn(0x7f64ec0fc030 msgr2=0x7f64ec0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.085+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 shutdown_connections 2024-03-16T10:26:00.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-03-16T10:26:00.086+0000 7f64f2a00640 1 -- 172.21.15.39:0/3829821939 wait complete. 2024-03-16T10:26:00.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:26:00.207 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:26:00.248 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:26:00.283 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-16T10:26:00.283 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T10:26:00.345 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T10:26:00.363 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T10:26:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:00 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:26:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:00 smithi105 ceph-mon[36559]: osdmap e120: 12 total, 12 up, 12 in 2024-03-16T10:26:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:00 smithi105 ceph-mon[36559]: from='client.? 172.21.15.39:0/3829821939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.364 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:00 smithi105 ceph-mon[36559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.380 INFO:tasks.cephadm:Stopping all daemons... 2024-03-16T10:26:00.380 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-16T10:26:00.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a 2024-03-16T10:26:00.455 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:00 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:26:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:00 smithi194 ceph-mon[36445]: osdmap e120: 12 total, 12 up, 12 in 2024-03-16T10:26:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:00 smithi194 ceph-mon[36445]: from='client.? 172.21.15.39:0/3829821939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.456 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:00 smithi194 ceph-mon[36445]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3193501938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:26:00.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-mon[31510]: osdmap e120: 12 total, 12 up, 12 in 2024-03-16T10:26:00.476 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-mon[31510]: from='client.? 172.21.15.39:0/3829821939' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.476 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-mon[31510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:26:00.476 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:26:00.140+0000 7f7152a00700 -1 client.25495: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-03-16T10:26:00.476 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:26:00.142+0000 7f7152a00700 -1 client.25495: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-03-16T10:26:00.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:00 smithi039 systemd[1]: Stopping Ceph mon.a for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:01.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a[31506]: 2024-03-16T10:26:01.071+0000 7f4938800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:01.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a[31506]: 2024-03-16T10:26:01.071+0000 7f4938800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-16T10:26:01.506 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:01 smithi039 podman[93490]: 2024-03-16 10:26:01.155681604 +0000 UTC m=+0.250951074 container died 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:26:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:01 smithi039 podman[93490]: 2024-03-16 10:26:01.655322554 +0000 UTC m=+0.750592078 container cleanup 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T10:26:01.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:01 smithi039 bash[93490]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a 2024-03-16T10:26:02.505 INFO:journalctl@ceph.mon.a.smithi039.stdout:Mar 16 10:26:02 smithi039 podman[93501]: 2024-03-16 10:26:02.157067124 +0000 UTC m=+0.987808076 container remove 40522528bdd28d57e662117530b8c219a732d08728bec31739850f120f1303fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T10:26:02.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.a.service' 2024-03-16T10:26:02.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:02.909 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-16T10:26:02.909 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-16T10:26:02.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.b 2024-03-16T10:26:03.365 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:02 smithi105 systemd[1]: Stopping Ceph mon.b for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:03 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-b[36555]: 2024-03-16T10:26:03.596+0000 7f010fada700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:03 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-b[36555]: 2024-03-16T10:26:03.596+0000 7f010fada700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-16T10:26:03.865 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:03 smithi105 podman[73960]: 2024-03-16 10:26:03.649401632 +0000 UTC m=+0.255853969 container died ef1478987f9410e3d741a6e5b6e4374ec9e54cec360b918612de2182320b048c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T10:26:07.486 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:07 smithi105 podman[73960]: 2024-03-16 10:26:07.151116252 +0000 UTC m=+3.757568596 container cleanup ef1478987f9410e3d741a6e5b6e4374ec9e54cec360b918612de2182320b048c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T10:26:07.486 INFO:journalctl@ceph.mon.b.smithi105.stdout:Mar 16 10:26:07 smithi105 bash[73960]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-b 2024-03-16T10:26:07.603 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.b.service' 2024-03-16T10:26:07.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:07.658 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-16T10:26:07.658 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-16T10:26:07.659 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.c 2024-03-16T10:26:08.066 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:07 smithi194 systemd[1]: Stopping Ceph mon.c for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:08.329 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c[36441]: 2024-03-16T10:26:08.064+0000 7f746163f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:08.329 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c[36441]: 2024-03-16T10:26:08.064+0000 7f746163f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-16T10:26:08.593 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:08 smithi194 podman[75116]: 2024-03-16 10:26:08.329509208 +0000 UTC m=+0.376192178 container died 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-16T10:26:08.858 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:08 smithi194 podman[75116]: 2024-03-16 10:26:08.840998388 +0000 UTC m=+0.887681351 container cleanup 03457543a46d52cfb27a11f5cba1a18c5d13f366e5ea79a54fffa4f1fbc3a68a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T10:26:08.858 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 10:26:08 smithi194 bash[75116]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mon-c 2024-03-16T10:26:09.232 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mon.c.service' 2024-03-16T10:26:09.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:09.294 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-16T10:26:09.294 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-16T10:26:09.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x 2024-03-16T10:26:09.754 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:09 smithi039 systemd[1]: Stopping Ceph mgr.x for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:10.258 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x[31749]: 2024-03-16T10:26:09.957+0000 7f71fee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:10.259 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:10 smithi039 podman[93703]: 2024-03-16 10:26:10.052596385 +0000 UTC m=+0.240775897 container died d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:26:10.755 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:10 smithi039 podman[93703]: 2024-03-16 10:26:10.536692318 +0000 UTC m=+0.724871823 container cleanup d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:26:10.756 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:10 smithi039 bash[93703]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x 2024-03-16T10:26:11.255 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Mar 16 10:26:10 smithi039 podman[93716]: 2024-03-16 10:26:10.80462609 +0000 UTC m=+0.724425531 container remove d8f20e15c4f33cabbd0591f3da53afe2128ddea537aa0a7a3a8ea21df3915b2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-16T10:26:11.394 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-16T10:26:11.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:11.525 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-16T10:26:11.525 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-16T10:26:11.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.y 2024-03-16T10:26:11.865 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:26:11 smithi105 systemd[1]: Stopping Ceph mgr.y for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:13.364 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:26:13 smithi105 podman[74110]: 2024-03-16 10:26:13.027056562 +0000 UTC m=+1.303115219 container died 09abc74760e283f2c7865a54b7fb65dbaa51f940bbb088f6f0866c4c6147b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T10:26:16.743 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:26:16 smithi105 podman[74110]: 2024-03-16 10:26:16.550149582 +0000 UTC m=+4.826208233 container cleanup 09abc74760e283f2c7865a54b7fb65dbaa51f940bbb088f6f0866c4c6147b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T10:26:16.744 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Mar 16 10:26:16 smithi105 bash[74110]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-y 2024-03-16T10:26:16.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-16T10:26:16.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:16.997 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-16T10:26:16.998 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-16T10:26:16.998 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.z 2024-03-16T10:26:17.341 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:26:17 smithi194 systemd[1]: Stopping Ceph mgr.z for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:17.637 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:26:17 smithi194 podman[75251]: 2024-03-16 10:26:17.341262835 +0000 UTC m=+0.146737745 container died 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T10:26:17.955 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:26:17 smithi194 podman[75251]: 2024-03-16 10:26:17.746214182 +0000 UTC m=+0.551689137 container cleanup 7eb5976ebe7545e69b18a3a35a00d049bc2546b648a3214cc8bbae640c17f23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:26:17.956 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 10:26:17 smithi194 bash[75251]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mgr-z 2024-03-16T10:26:18.170 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-16T10:26:18.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:18.211 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-16T10:26:18.211 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-16T10:26:18.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.0 2024-03-16T10:26:18.755 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:18 smithi039 systemd[1]: Stopping Ceph osd.0 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:19.255 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0[46558]: 2024-03-16T10:26:18.814+0000 7fce5c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:19.255 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0[46558]: 2024-03-16T10:26:18.814+0000 7fce5c000700 -1 osd.0 121 *** Got signal Terminated *** 2024-03-16T10:26:19.255 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0[46558]: 2024-03-16T10:26:18.815+0000 7fce5c000700 -1 osd.0 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:26:25.506 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:25 smithi039 podman[93823]: 2024-03-16 10:26:25.154186627 +0000 UTC m=+6.507793488 container died 15d0c9408d9fb776592d0880b412d05de4f6676560ffe9680a7bd3b4cd93fc5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T10:26:26.001 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:25 smithi039 podman[93823]: 2024-03-16 10:26:25.749891938 +0000 UTC m=+7.103498843 container cleanup 15d0c9408d9fb776592d0880b412d05de4f6676560ffe9680a7bd3b4cd93fc5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T10:26:26.001 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:25 smithi039 bash[93823]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0 2024-03-16T10:26:26.255 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:26 smithi039 podman[93858]: 2024-03-16 10:26:26.000462838 +0000 UTC m=+0.823731659 container remove 15d0c9408d9fb776592d0880b412d05de4f6676560ffe9680a7bd3b4cd93fc5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-16T10:26:27.249 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:26 smithi039 podman[93946]: 2024-03-16 10:26:26.808960332 +0000 UTC m=+0.043988032 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:26:27.249 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:26 smithi039 podman[93946]: 2024-03-16 10:26:26.952217563 +0000 UTC m=+0.187245286 container create cddd63587862c56713846c642f8007edbcd2461f69a0879fd6381e344921dfe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True) 2024-03-16T10:26:27.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:27 smithi039 podman[93946]: 2024-03-16 10:26:27.248210697 +0000 UTC m=+0.483238474 container init cddd63587862c56713846c642f8007edbcd2461f69a0879fd6381e344921dfe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T10:26:27.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:27 smithi039 podman[93946]: 2024-03-16 10:26:27.25928056 +0000 UTC m=+0.494308250 container start cddd63587862c56713846c642f8007edbcd2461f69a0879fd6381e344921dfe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T10:26:27.505 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:27 smithi039 podman[93946]: 2024-03-16 10:26:27.479880014 +0000 UTC m=+0.714907771 container attach cddd63587862c56713846c642f8007edbcd2461f69a0879fd6381e344921dfe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T10:26:28.755 INFO:journalctl@ceph.osd.0.smithi039.stdout:Mar 16 10:26:28 smithi039 podman[93946]: 2024-03-16 10:26:28.314738194 +0000 UTC m=+1.549765892 container died cddd63587862c56713846c642f8007edbcd2461f69a0879fd6381e344921dfe7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:26:29.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.0.service' 2024-03-16T10:26:29.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:29.486 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-16T10:26:29.487 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-16T10:26:29.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.1 2024-03-16T10:26:29.796 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:29 smithi105 systemd[1]: Stopping Ceph osd.1 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:30.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1[41146]: 2024-03-16T10:26:29.795+0000 7fe0f324b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:30.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1[41146]: 2024-03-16T10:26:29.795+0000 7fe0f324b700 -1 osd.1 121 *** Got signal Terminated *** 2024-03-16T10:26:30.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1[41146]: 2024-03-16T10:26:29.795+0000 7fe0f324b700 -1 osd.1 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:26:35.815 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:35 smithi105 podman[74242]: 2024-03-16 10:26:35.505844718 +0000 UTC m=+5.830253539 container died 80fc50a11e1eecb6600cda3c8f7fb6a5df256650ea2670783e81888a189b096e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T10:26:36.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:35 smithi105 podman[74242]: 2024-03-16 10:26:35.904635617 +0000 UTC m=+6.229044423 container cleanup 80fc50a11e1eecb6600cda3c8f7fb6a5df256650ea2670783e81888a189b096e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T10:26:36.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:35 smithi105 bash[74242]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1 2024-03-16T10:26:36.563 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:36 smithi105 podman[74260]: 2024-03-16 10:26:36.206182584 +0000 UTC m=+0.672703032 container remove 80fc50a11e1eecb6600cda3c8f7fb6a5df256650ea2670783e81888a189b096e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T10:26:36.860 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:36 smithi105 podman[74352]: 2024-03-16 10:26:36.465801165 +0000 UTC m=+0.017706531 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:26:36.860 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:36 smithi105 podman[74352]: 2024-03-16 10:26:36.5988823 +0000 UTC m=+0.150787644 container create dbac14d22928af8d2df221148ca6c169d201ce7aace8669de59853ce713425d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:26:37.114 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:36 smithi105 podman[74352]: 2024-03-16 10:26:36.860378289 +0000 UTC m=+0.412283642 container init dbac14d22928af8d2df221148ca6c169d201ce7aace8669de59853ce713425d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T10:26:37.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:36 smithi105 podman[74352]: 2024-03-16 10:26:36.863746328 +0000 UTC m=+0.415651685 container start dbac14d22928af8d2df221148ca6c169d201ce7aace8669de59853ce713425d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T10:26:37.115 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:37 smithi105 podman[74352]: 2024-03-16 10:26:37.035675251 +0000 UTC m=+0.587580619 container attach dbac14d22928af8d2df221148ca6c169d201ce7aace8669de59853ce713425d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:26:37.966 INFO:journalctl@ceph.osd.1.smithi105.stdout:Mar 16 10:26:37 smithi105 podman[74352]: 2024-03-16 10:26:37.703323165 +0000 UTC m=+1.255228527 container died dbac14d22928af8d2df221148ca6c169d201ce7aace8669de59853ce713425d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:26:38.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.1.service' 2024-03-16T10:26:38.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:38.353 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-16T10:26:38.353 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-16T10:26:38.353 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.2 2024-03-16T10:26:38.675 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:38 smithi194 systemd[1]: Stopping Ceph osd.2 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:38.934 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:26:38.673+0000 7fdb68fdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:38.934 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:26:38.673+0000 7fdb68fdf700 -1 osd.2 121 *** Got signal Terminated *** 2024-03-16T10:26:38.934 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2[41716]: 2024-03-16T10:26:38.673+0000 7fdb68fdf700 -1 osd.2 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:26:44.801 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:44 smithi194 podman[75374]: 2024-03-16 10:26:44.4845705 +0000 UTC m=+5.925364644 container died 6f281fce8970897d404e4b85d70fe99bcda24e633b5c41e6c6623d90fdd6f965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-03-16T10:26:45.177 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:44 smithi194 podman[75374]: 2024-03-16 10:26:44.899732454 +0000 UTC m=+6.340526595 container cleanup 6f281fce8970897d404e4b85d70fe99bcda24e633b5c41e6c6623d90fdd6f965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-03-16T10:26:45.177 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:44 smithi194 bash[75374]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2 2024-03-16T10:26:45.456 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75405]: 2024-03-16 10:26:45.177541536 +0000 UTC m=+0.649207004 container remove 6f281fce8970897d404e4b85d70fe99bcda24e633b5c41e6c6623d90fdd6f965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T10:26:45.772 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75499]: 2024-03-16 10:26:45.392321958 +0000 UTC m=+0.018075191 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:26:45.772 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75499]: 2024-03-16 10:26:45.518924451 +0000 UTC m=+0.144677694 container create 8bcadd2f5d5370509ea7b368de21d23ce36f15f82d6b908b379f056faae59c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-16T10:26:45.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:45.616+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:46.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75499]: 2024-03-16 10:26:45.772004822 +0000 UTC m=+0.397758056 container init 8bcadd2f5d5370509ea7b368de21d23ce36f15f82d6b908b379f056faae59c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T10:26:46.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75499]: 2024-03-16 10:26:45.775527566 +0000 UTC m=+0.401280794 container start 8bcadd2f5d5370509ea7b368de21d23ce36f15f82d6b908b379f056faae59c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-16T10:26:46.206 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:45 smithi194 podman[75499]: 2024-03-16 10:26:45.930341177 +0000 UTC m=+0.556094486 container attach 8bcadd2f5d5370509ea7b368de21d23ce36f15f82d6b908b379f056faae59c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T10:26:46.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:46.453+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:46.884 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 10:26:46 smithi194 podman[75499]: 2024-03-16 10:26:46.508938638 +0000 UTC m=+1.134691864 container died 8bcadd2f5d5370509ea7b368de21d23ce36f15f82d6b908b379f056faae59c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T10:26:46.884 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:46.629+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:47.226 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.2.service' 2024-03-16T10:26:47.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:47.273 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-16T10:26:47.273 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-16T10:26:47.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.3 2024-03-16T10:26:47.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:47.413+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:47.755 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:47 smithi039 systemd[1]: Stopping Ceph osd.3 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:47.958 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:47.618+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:47.958 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:47.492+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:48.255 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3[53294]: 2024-03-16T10:26:47.842+0000 7fbb60e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:48.255 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3[53294]: 2024-03-16T10:26:47.842+0000 7fbb60e00700 -1 osd.3 121 *** Got signal Terminated *** 2024-03-16T10:26:48.256 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3[53294]: 2024-03-16T10:26:47.842+0000 7fbb60e00700 -1 osd.3 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:26:48.614 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:48.262+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:48.646 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:48 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:48.393+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:48.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:48.647+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:48.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:48.527+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:49.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:49.047+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:49.615 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:49.302+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:49.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:49 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:49.415+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:49.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:49 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:49.654+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:49.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:49.662+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:49.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:49.534+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:50.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:50.081+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:50.614 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:50.347+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:50.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:50 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:50.416+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:50.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:50 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:50.665+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:50.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:50.666+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:50.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:50.570+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:51.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:51.040+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:51.614 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:51.344+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:51.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:51 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:51.456+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:51.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:51 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:51.657+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:51.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:51.620+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:51.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:51.597+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:52.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:52.033+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:52.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:52 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:52.408+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:52.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:52 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:52.627+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:52.864 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:52.373+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:52.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:52.587+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:52.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:52.571+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:53.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:53.036+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:53.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:53 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:53.430+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:53.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:53 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:53.672+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:53.864 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:53.401+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:53.961 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:53.615+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:53.961 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:53.621+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:54.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:54.001+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:54.708 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:54 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:54.389+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:54.709 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:54 smithi039 podman[94052]: 2024-03-16 10:26:54.50661154 +0000 UTC m=+6.809995437 container died 26de97dd3690c3a16272594003bae872342db1b1ff7500ed31695961cf404a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T10:26:54.864 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:54.414+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:54.955 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:54.634+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:54.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:54.663+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:54.990 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:54 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:54.706+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:55.255 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:55 smithi039 podman[94052]: 2024-03-16 10:26:55.144557038 +0000 UTC m=+7.447940931 container cleanup 26de97dd3690c3a16272594003bae872342db1b1ff7500ed31695961cf404a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:26:55.255 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:55 smithi039 bash[94052]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3 2024-03-16T10:26:55.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:55.032+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:55.680 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:55 smithi039 podman[94083]: 2024-03-16 10:26:55.345916477 +0000 UTC m=+0.806129444 container remove 26de97dd3690c3a16272594003bae872342db1b1ff7500ed31695961cf404a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:26:55.681 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:55 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:55.361+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:55.864 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:55.387+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:55.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:55.668+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:55.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:55.667+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:56.004 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:55 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:55.678+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:56.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:56.056+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:56.505 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:56.322+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:56.505 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:56 smithi039 podman[94169]: 2024-03-16 10:26:56.137288632 +0000 UTC m=+0.033165989 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:26:56.505 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:56 smithi039 podman[94169]: 2024-03-16 10:26:56.271688157 +0000 UTC m=+0.167565521 container create 4c76bbf73640c13cf8078ca12075dc8805cf35ea4a1214cada1ab784f498aada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T10:26:56.864 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:56.429+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:56.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:26:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:26:56.515+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:26:56.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:56.682+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:56.958 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:56.660+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:56.958 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:56.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:26:57.004 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:56.726+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:57.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:56 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:56.726+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:26:57.005 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:56 smithi039 podman[94169]: 2024-03-16 10:26:56.58420928 +0000 UTC m=+0.480086718 container init 4c76bbf73640c13cf8078ca12075dc8805cf35ea4a1214cada1ab784f498aada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-16T10:26:57.005 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:56 smithi039 podman[94169]: 2024-03-16 10:26:56.596189318 +0000 UTC m=+0.492066719 container start 4c76bbf73640c13cf8078ca12075dc8805cf35ea4a1214cada1ab784f498aada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-16T10:26:57.005 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:56 smithi039 podman[94169]: 2024-03-16 10:26:56.699618109 +0000 UTC m=+0.595495475 container attach 4c76bbf73640c13cf8078ca12075dc8805cf35ea4a1214cada1ab784f498aada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-16T10:26:57.402 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:57 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:57.285+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:57.402 INFO:journalctl@ceph.osd.3.smithi039.stdout:Mar 16 10:26:57 smithi039 podman[94169]: 2024-03-16 10:26:57.04314377 +0000 UTC m=+0.939021171 container died 4c76bbf73640c13cf8078ca12075dc8805cf35ea4a1214cada1ab784f498aada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:26:57.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:57.075+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:57.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.3.service' 2024-03-16T10:26:57.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:26:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:26:57.533+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:26:57.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:57.439+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:57.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:26:57.905 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-16T10:26:57.905 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-16T10:26:57.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.4 2024-03-16T10:26:57.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:57.707+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:57.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:57.647+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:57.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:57.647+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:26:58.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:57 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:57.749+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:58.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:57 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:57.749+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:26:58.212 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:57 smithi105 systemd[1]: Stopping Ceph osd.4 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:26:58.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:58.098+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:58.473 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:26:58.211+0000 7fa7b24d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:26:58.473 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:26:58.211+0000 7fa7b24d2700 -1 osd.4 121 *** Got signal Terminated *** 2024-03-16T10:26:58.473 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:26:58.211+0000 7fa7b24d2700 -1 osd.4 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:26:58.505 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:58 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:58.236+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:58.505 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:58 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:58.236+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:26:58.864 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:26:58.588+0000 7fa7aeac6700 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.598217+0000 front 2024-03-16T10:26:32.598263+0000 (oldest deadline 2024-03-16T10:26:58.498290+0000) 2024-03-16T10:26:58.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:26:58.560+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:26:58.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:58 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:58.472+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:58.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:58.679+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:58.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:58.599+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:58.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:58.599+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:26:59.006 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:58 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:58.700+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:26:59.006 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:58 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:58.700+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:26:59.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:59.100+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:26:59.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:26:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:26:59.100+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:26:59.684 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:59 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:59.256+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:26:59.685 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:26:59 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:26:59.256+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:26:59.865 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:26:59 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:26:59.582+0000 7fa7aeac6700 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.598217+0000 front 2024-03-16T10:26:32.598263+0000 (oldest deadline 2024-03-16T10:26:58.498290+0000) 2024-03-16T10:26:59.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:26:59 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:26:59.555+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:26:59.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:59 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:59.518+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:26:59.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:26:59 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:26:59.518+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:26:59.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:26:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:26:59.716+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:26:59.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:59.648+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:26:59.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:26:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:26:59.648+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:00.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:59 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:59.682+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:00.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:26:59 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:26:59.682+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:00.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:00.118+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:00.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:00.118+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:00.668 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:00.280+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:00.668 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:00.280+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:00.864 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:00 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:27:00.538+0000 7fa7aeac6700 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.598217+0000 front 2024-03-16T10:26:32.598263+0000 (oldest deadline 2024-03-16T10:26:58.498290+0000) 2024-03-16T10:27:00.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:00 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:00.548+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:00.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:00 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:00.546+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:00.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:00 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:00.546+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:00.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:00.702+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:00.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:00.702+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:00.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:00.681+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:00.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:00.681+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:01.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:00.666+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:01.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:00 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:00.666+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:01.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:01.116+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:01.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:01.116+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:01.641 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:01.307+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:01.641 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:01.307+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:01.864 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:01 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:27:01.495+0000 7fa7aeac6700 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.598217+0000 front 2024-03-16T10:26:32.598263+0000 (oldest deadline 2024-03-16T10:26:58.498290+0000) 2024-03-16T10:27:01.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:01 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:01.515+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:01.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:01 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:01.569+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:01.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:01 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:01.569+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:01.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:01.695+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:01.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:01.695+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:01.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:01.706+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:01.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:01.706+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:02.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:01.639+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:02.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:01 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:01.639+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:02.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:02.109+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:02.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:02.109+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:02.621 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:02.337+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:02.621 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:02.337+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:02.865 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:02 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4[47861]: 2024-03-16T10:27:02.488+0000 7fa7aeac6700 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.598217+0000 front 2024-03-16T10:26:32.598263+0000 (oldest deadline 2024-03-16T10:26:58.498290+0000) 2024-03-16T10:27:02.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:02 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:02.561+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:02.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:02 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:02.594+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:02.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:02 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:02.594+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:02.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:02.740+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:02.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:02.740+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:02.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:02.703+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:02.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:02.703+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:03.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:02.619+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:03.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:02 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:02.619+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:03.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:03.137+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:03.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:03.137+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:03.665 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:03.385+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:03.665 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:03.385+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:03.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:03 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:03.539+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:03.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:03 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:03.569+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:03.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:03 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:03.569+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:03.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:03.730+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:03.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:03.730+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:03.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:03.698+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:03.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:03.698+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:04.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:03.663+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:04.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:03 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:03.663+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:04.256 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:03 smithi105 podman[74473]: 2024-03-16 10:27:03.968797588 +0000 UTC m=+5.871076040 container died 45828de03626307cc139061e9e462959d8989ce61bfa5c334d1d073d63ca4ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T10:27:04.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:04.173+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:04.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:04.173+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:04.561 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:04 smithi105 podman[74473]: 2024-03-16 10:27:04.362365502 +0000 UTC m=+6.264643982 container cleanup 45828de03626307cc139061e9e462959d8989ce61bfa5c334d1d073d63ca4ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:27:04.561 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:04 smithi105 bash[74473]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4 2024-03-16T10:27:04.702 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:04.387+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:04.702 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:04.387+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:04.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:04.559+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:04.865 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:04 smithi105 podman[74490]: 2024-03-16 10:27:04.621930524 +0000 UTC m=+0.640673098 container remove 45828de03626307cc139061e9e462959d8989ce61bfa5c334d1d073d63ca4ebe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T10:27:04.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:04.566+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:04.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:04 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:04.566+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:04.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:04.768+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:04.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:04.768+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:04.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:04.654+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:04.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:04.654+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:05.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:04.701+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:05.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:04.701+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:05.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:04 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:04.701+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:05.234 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:04 smithi105 podman[74581]: 2024-03-16 10:27:04.841820215 +0000 UTC m=+0.025657198 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:05.234 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:04 smithi105 podman[74581]: 2024-03-16 10:27:04.98152132 +0000 UTC m=+0.165358290 container create 1f9efffebbe1393864d11c8e116d51358bec016c4680499ce4d9946c4ec07b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-16T10:27:05.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:05.202+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:05.456 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:05.202+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:05.535 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:05 smithi105 podman[74581]: 2024-03-16 10:27:05.234076657 +0000 UTC m=+0.417913641 container init 1f9efffebbe1393864d11c8e116d51358bec016c4680499ce4d9946c4ec07b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T10:27:05.535 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:05 smithi105 podman[74581]: 2024-03-16 10:27:05.237394669 +0000 UTC m=+0.421231641 container start 1f9efffebbe1393864d11c8e116d51358bec016c4680499ce4d9946c4ec07b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:27:05.535 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:05 smithi105 podman[74581]: 2024-03-16 10:27:05.317356585 +0000 UTC m=+0.501193569 container attach 1f9efffebbe1393864d11c8e116d51358bec016c4680499ce4d9946c4ec07b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T10:27:05.726 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:05.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:05.726 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:05.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:05.846 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:05.595+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:05.846 INFO:journalctl@ceph.osd.4.smithi105.stdout:Mar 16 10:27:05 smithi105 podman[74581]: 2024-03-16 10:27:05.535306243 +0000 UTC m=+0.719143224 container died 1f9efffebbe1393864d11c8e116d51358bec016c4680499ce4d9946c4ec07b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:27:05.846 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:05.544+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:05.846 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:05 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:05.544+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:05.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:05.804+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:05.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:05.804+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:05.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:05.652+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:05.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:05.652+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:06.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:05.724+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:06.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:05.724+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:06.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:05 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:05.724+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:06.156 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.4.service' 2024-03-16T10:27:06.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:06.197 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-16T10:27:06.197 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-16T10:27:06.197 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.5 2024-03-16T10:27:06.478 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 systemd[1]: Stopping Ceph osd.5 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:06.479 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:06.249+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:06.479 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:06.249+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:06.479 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:06.477+0000 7f5cbecf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:06.479 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:06.477+0000 7f5cbecf0700 -1 osd.5 121 *** Got signal Terminated *** 2024-03-16T10:27:06.479 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:06.477+0000 7f5cbecf0700 -1 osd.5 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:06.748 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:06.368+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:06.748 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:06.368+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:06.852 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:06.659+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:06.852 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:06.659+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:06.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:06.591+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:06.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:06.584+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:06.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:06.584+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:06.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:06 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:06.584+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:07.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:06.747+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:07.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:06.747+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:07.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:06 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:06.747+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:07.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:06.851+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:07.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:06 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:06.851+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:07.653 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:07.236+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:07.654 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:07.236+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:07.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:07.411+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:07.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:07.411+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:07.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:07 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:07.559+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:07.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:07 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:07.590+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:07.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:07 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:07.590+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:07.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:07 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:07.590+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:07.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:07.847+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:07.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:07.847+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:07.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:07.847+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:07.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:07.652+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:07.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:07 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:07.652+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:08.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:07.792+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:08.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:07.792+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:08.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:07 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:07.792+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:08.604 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:08.260+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:08.604 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:08.260+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:08.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:08.408+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:08.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:08.408+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:08.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:08.408+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:08.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:08.541+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:08.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:08.541+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:08.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:08.594+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:08.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:08.594+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:08.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:08 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:08.594+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:08.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:08.844+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:08.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:08.844+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:08.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:08.844+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:08.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:08.603+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:08.957 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:08 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:08.603+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:09.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:08.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:09.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:08.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:09.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:08 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:08.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:09.627 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:09.234+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:09.627 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:09.234+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:09.627 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:09.234+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.541422+0000 front 2024-03-16T10:26:43.541448+0000 (oldest deadline 2024-03-16T10:27:08.841550+0000) 2024-03-16T10:27:09.739 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:09.457+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:09.739 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:09.457+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:09.739 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:09.457+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:09.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:09.538+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:09.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:09.538+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:09.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:09.633+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:09.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:09.633+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:09.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:09 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:09.633+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:09.955 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:09.626+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:09.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:09.626+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:09.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:09.626+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:09.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:09.831+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:09.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:09.831+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:09.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:09 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:09.831+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:10.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:09.737+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:10.006 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:09.737+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:10.006 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:09.737+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:10.652 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:10.260+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:10.653 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:10.260+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:10.653 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:10.260+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.541422+0000 front 2024-03-16T10:26:43.541448+0000 (oldest deadline 2024-03-16T10:27:08.841550+0000) 2024-03-16T10:27:10.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:10.415+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:10.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:10.415+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:10.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:10.415+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:10.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:10.514+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:10.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:10.514+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:10.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:10.599+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:10.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:10.599+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:10.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:10 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:10.599+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:10.651+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:10.651+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:10.651+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:10.802+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:10.802+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:10.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:10 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:10.802+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:11.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:10.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:11.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:10.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:11.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:10 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:10.783+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:11.662 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:11.223+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.737837+0000 front 2024-03-16T10:26:22.737741+0000 (oldest deadline 2024-03-16T10:26:48.037603+0000) 2024-03-16T10:27:11.663 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:11.223+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:32.639447+0000 front 2024-03-16T10:26:32.639266+0000 (oldest deadline 2024-03-16T10:26:58.539139+0000) 2024-03-16T10:27:11.663 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5[48425]: 2024-03-16T10:27:11.223+0000 7f5cbb2e4700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.541422+0000 front 2024-03-16T10:26:43.541448+0000 (oldest deadline 2024-03-16T10:27:08.841550+0000) 2024-03-16T10:27:11.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:11.448+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:11.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:11.448+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:11.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:11.448+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:11.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:11.564+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:11.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:11.564+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:11.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:11.649+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:11.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:11.649+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:11.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:11 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:11.649+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:11.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:11.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:11.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:11.800+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:11.800+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:11.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:11 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:11.800+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:12.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:11.767+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:12.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:11.767+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:12.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:11 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:11.767+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:12.706 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:12 smithi194 podman[75603]: 2024-03-16 10:27:12.450092289 +0000 UTC m=+6.064888269 container died a3db471dc753b15b154a5ab109221243ad4e386e1a74aa7e0c37f2c76252f9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-16T10:27:12.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:12.695+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:12.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:12.695+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:12.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:12.695+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:12.720 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:12.403+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:12.720 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:12.403+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:12.720 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:12.403+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:12.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:12.574+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:12.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:12.574+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:12.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:12.672+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:12.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:12.672+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:12.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:12 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:12.672+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:13.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:12.719+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:13.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:12.719+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:13.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:12 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:12.719+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:13.138 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:12 smithi194 podman[75603]: 2024-03-16 10:27:12.903458068 +0000 UTC m=+6.518254056 container cleanup a3db471dc753b15b154a5ab109221243ad4e386e1a74aa7e0c37f2c76252f9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:27:13.138 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:12 smithi194 bash[75603]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5 2024-03-16T10:27:13.138 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:12.783+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:13.138 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:12.783+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:13.138 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:12 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:12.783+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:13.455 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75620]: 2024-03-16 10:27:13.137957526 +0000 UTC m=+0.661004764 container remove a3db471dc753b15b154a5ab109221243ad4e386e1a74aa7e0c37f2c76252f9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:27:13.706 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:13.409+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:13.706 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:13.409+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:13.706 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:13.409+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:13.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:13.700+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:13.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:13.700+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:13.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:13.700+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:13.831 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75712]: 2024-03-16 10:27:13.445309929 +0000 UTC m=+0.018152708 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:13.831 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75712]: 2024-03-16 10:27:13.572038677 +0000 UTC m=+0.144881440 container create 1f98df70e79653fbe009775d51a879db5e9d1c72cf97ea65b730638605ea3186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T10:27:13.831 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75712]: 2024-03-16 10:27:13.774455192 +0000 UTC m=+0.347297962 container init 1f98df70e79653fbe009775d51a879db5e9d1c72cf97ea65b730638605ea3186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-16T10:27:13.831 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75712]: 2024-03-16 10:27:13.777708168 +0000 UTC m=+0.350550932 container start 1f98df70e79653fbe009775d51a879db5e9d1c72cf97ea65b730638605ea3186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-16T10:27:13.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:13 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:13.591+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:13.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:13 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:13.591+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:13.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:13 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:13.686+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:13.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:13 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:13.686+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:13.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:13 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:13.686+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:14.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:13.704+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:14.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:13.704+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:14.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:13 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:13.704+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:14.206 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:13 smithi194 podman[75712]: 2024-03-16 10:27:13.874461467 +0000 UTC m=+0.447304232 container attach 1f98df70e79653fbe009775d51a879db5e9d1c72cf97ea65b730638605ea3186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-16T10:27:14.207 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 10:27:14 smithi194 podman[75712]: 2024-03-16 10:27:14.078486841 +0000 UTC m=+0.651329614 container died 1f98df70e79653fbe009775d51a879db5e9d1c72cf97ea65b730638605ea3186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:27:14.207 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:13.829+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:14.207 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:13.829+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:14.207 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:13 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:13.829+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:14.755 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.5.service' 2024-03-16T10:27:14.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:14.451+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:14.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:14.451+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:14.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:14.451+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:14.757 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:14.656+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:14.757 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:14.656+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:14.757 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:14.656+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:14.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:14.798 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-16T10:27:14.798 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-16T10:27:14.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.6 2024-03-16T10:27:14.864 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:14 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:14.598+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:14.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:14 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:14.598+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:14.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:14 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:14.654+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:14.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:14 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:14.654+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:14.865 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:14 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:14.654+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:14.867+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:14.867+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:14.867+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:14.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:14.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:14.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:14 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:14.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:15.255 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:14 smithi039 systemd[1]: Stopping Ceph osd.6 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:15.662 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.387+0000 7fbbe1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:15.662 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.387+0000 7fbbe1400700 -1 osd.6 121 *** Got signal Terminated *** 2024-03-16T10:27:15.663 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.387+0000 7fbbe1400700 -1 osd.6 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:15.663 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.407+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:15.663 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.407+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:15.663 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.407+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:15.663 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:15.407+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:16.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:15.659+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:16.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:15.659+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:16.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:15 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:15.659+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:16.114 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:15.618+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:16.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:15.618+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:16.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:15.694+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:16.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:15.694+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:16.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:15.694+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:16.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:15 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:15.694+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:15.869+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:15.869+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:15.869+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:15.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:15.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:16.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:15 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:15.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:16.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:16.640+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:16.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:16.640+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:16.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:16.640+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:16.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:16.428+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:16.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:16.428+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:16.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:16.428+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:16.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:16 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:16.428+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:17.114 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:16.646+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:17.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:16.646+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:17.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:16.735+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:17.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:16.735+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:17.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:16.735+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:17.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:16 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:16.735+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:16.827+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:16.827+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:16.827+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:16.757+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:16.757+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:17.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:16 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:16.757+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:17.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:17.649+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:17.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:17.649+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:17.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:17.649+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:17.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:17.405+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:17.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:17.405+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:17.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:17.405+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:17.756 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:17 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:17.405+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:18.016 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:17.633+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:18.016 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:17.633+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:18.016 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:17.633+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:18.016 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:17.783+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:18.016 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:17.783+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:18.017 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:17.783+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:18.017 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:17 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:17.783+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:17.787+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:17.787+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:17.787+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:17.731+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:17.731+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:18.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:17 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:17.731+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:18.620+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:18.620+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:18.620+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:18.445+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:18.445+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:18.445+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:18.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:18.445+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:18.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:18.744+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:18.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:18.744+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:18.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:18.744+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:18.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:18.684+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:18.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:18.684+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:18.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:18 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:18.684+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:19.114 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:18.621+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:18.621+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:18.621+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:18.817+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:18.817+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:18.817+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:19.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:18 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:18.817+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:19.687 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:19.597+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:19.687 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:19.597+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:19.687 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:19.597+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:19.687 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:19.399+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:19.687 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:19.399+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:19.688 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:19.399+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:19.688 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:19 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:19.399+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:19.727+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:19.727+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:19.727+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:19.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:19.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:19.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:19 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:19.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:20.114 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:19.630+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:19.630+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:19.630+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:19.852+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:19.852+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:19.852+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:20.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:19 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:19.852+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:20.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.266919+0000 front 2024-03-16T10:26:22.267625+0000 (oldest deadline 2024-03-16T10:26:46.366747+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:20.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.370770+0000 front 2024-03-16T10:26:33.370571+0000 (oldest deadline 2024-03-16T10:26:57.971841+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:20.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.375112+0000 front 2024-03-16T10:26:41.375172+0000 (oldest deadline 2024-03-16T10:27:08.376803+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6[60136]: 2024-03-16T10:27:20.354+0000 7fbbd7c00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:50.580277+0000 front 2024-03-16T10:26:50.580336+0000 (oldest deadline 2024-03-16T10:27:15.280075+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:20.574+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:20.574+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:20.574+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:20.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:20 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:20.574+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:20.726+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:20.726+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:20.726+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:20.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:20.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:20.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:20 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:20.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:20.642+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:20.642+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:20.642+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:20.881+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:20.881+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:20.881+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:21.118 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:20 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:20.881+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:21.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:21.600+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:21.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:21.600+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:21.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:21.600+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:22.005 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:21 smithi039 podman[94278]: 2024-03-16 10:27:21.756064281 +0000 UTC m=+6.508089857 container died e6c196c1ee845c49fcbba27c1a9c9c53fe99b2028c3703dbdba923b5e0a96d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T10:27:22.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:21 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:21.622+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:22.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:21 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:21.622+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:22.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:21 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:21.622+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:22.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:21 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:21.622+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:21.728+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:21.728+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:21.728+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:21.718+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:21.718+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:22.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:21 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:21.718+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:22.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:21.876+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:22.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:21.876+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:22.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:21.876+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:22.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:21 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:21.876+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:22.491 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:22 smithi039 podman[94278]: 2024-03-16 10:27:22.247392067 +0000 UTC m=+6.999417695 container cleanup e6c196c1ee845c49fcbba27c1a9c9c53fe99b2028c3703dbdba923b5e0a96d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T10:27:22.491 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:22 smithi039 bash[94278]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6 2024-03-16T10:27:22.755 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:22 smithi039 podman[94310]: 2024-03-16 10:27:22.490462986 +0000 UTC m=+0.700847160 container remove e6c196c1ee845c49fcbba27c1a9c9c53fe99b2028c3703dbdba923b5e0a96d4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T10:27:22.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:22.581+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:22.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:22.581+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:22.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:22.581+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:22.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:22.581+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:22.915 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:22.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:22.916 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:22.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:22.916 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:22.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:23.205 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:22.753+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:23.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:22.753+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:23.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:22.753+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:23.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:22.750+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:23.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:22.750+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:23.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:22 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:22.750+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:23.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:22.914+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:23.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:22.914+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:23.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:22.914+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:23.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:22 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:22.914+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:23.620 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:23 smithi039 podman[94400]: 2024-03-16 10:27:23.253533382 +0000 UTC m=+0.040497632 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:23.620 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:23 smithi039 podman[94400]: 2024-03-16 10:27:23.349572478 +0000 UTC m=+0.136536742 container create d6e52042aadd0ad52e60e121ce4765d221ce8ad1b2f0c0391a0fded7c07566b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:27:23.924 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:23.660+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:23.924 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:23.660+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:23.924 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:23.660+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:24.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:23 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:23.617+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:24.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:23 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:23.618+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:24.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:23 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:23.618+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:24.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:23 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:23.618+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:24.006 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:23 smithi039 podman[94400]: 2024-03-16 10:27:23.66146606 +0000 UTC m=+0.448430328 container init d6e52042aadd0ad52e60e121ce4765d221ce8ad1b2f0c0391a0fded7c07566b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:27:24.006 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:23 smithi039 podman[94400]: 2024-03-16 10:27:23.673776338 +0000 UTC m=+0.460740606 container start d6e52042aadd0ad52e60e121ce4765d221ce8ad1b2f0c0391a0fded7c07566b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:27:24.006 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:23 smithi039 podman[94400]: 2024-03-16 10:27:23.751856596 +0000 UTC m=+0.538820900 container attach d6e52042aadd0ad52e60e121ce4765d221ce8ad1b2f0c0391a0fded7c07566b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:23.712+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:23.712+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:23.712+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:23.799+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:23.799+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:24.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:23 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:23.799+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:24.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:23.923+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:24.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:23.923+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:24.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:23.923+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:24.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:23 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:23.923+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:24.451 INFO:journalctl@ceph.osd.6.smithi039.stdout:Mar 16 10:27:24 smithi039 podman[94400]: 2024-03-16 10:27:24.096102736 +0000 UTC m=+0.883066987 container died d6e52042aadd0ad52e60e121ce4765d221ce8ad1b2f0c0391a0fded7c07566b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:27:24.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:24 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:24.614+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:24.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:24 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:24.614+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:24.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:24 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:24.615+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:24.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:24 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:24.615+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:24.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.6.service' 2024-03-16T10:27:24.932 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:24.623+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:24.932 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:24.623+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:24.932 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:24.623+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:24.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:24.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:24.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:24.663+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:24.663+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:24.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:24 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:24.663+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:24.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:24.992 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-16T10:27:24.992 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-16T10:27:24.992 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.7 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 systemd[1]: Stopping Ceph osd.7 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:24.930+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:24.930+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:24.930+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:24.930+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:25.318 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:24 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:24.930+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:25.601 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.316+0000 7f2a90c6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:25.601 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.316+0000 7f2a90c6b700 -1 osd.7 121 *** Got signal Terminated *** 2024-03-16T10:27:25.601 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.316+0000 7f2a90c6b700 -1 osd.7 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:25.602 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.599+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:25.602 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.599+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:25.602 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.599+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:25.602 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:25.599+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.691155+0000 front 2024-03-16T10:27:00.691128+0000 (oldest deadline 2024-03-16T10:27:25.391145+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:25.636+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:25.636+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:25.636+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:25.804+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:25.804+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:25.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:25 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:25.804+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:26.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:25 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:25.567+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:26.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:25 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:25.567+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:26.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:25 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:25.567+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:26.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:25 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:25.567+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:26.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:25.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:26.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:25.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:26.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:25.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:26.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:25.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:26.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:25.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:26.929 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:26.641+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:26.929 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:26.641+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:26.929 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:26.641+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:26.929 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:26.641+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.691155+0000 front 2024-03-16T10:27:00.691128+0000 (oldest deadline 2024-03-16T10:27:25.391145+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:26.683+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:26.683+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:26.683+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:26.798+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:26.798+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:26.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:26 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:26.798+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:27.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:26 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:26.596+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:27.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:26 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:26.596+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:27.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:26 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:26.596+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:27.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:26 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:26.596+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:27.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:26.927+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:27.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:26.927+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:27.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:26.927+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:27.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:26.927+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:27.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:26 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:26.927+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:27.947 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:27.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:27.947 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:27.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:27.948 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:27.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:27.948 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:27.629+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.691155+0000 front 2024-03-16T10:27:00.691128+0000 (oldest deadline 2024-03-16T10:27:25.391145+0000) 2024-03-16T10:27:27.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:27.687+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:27.955 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:27.687+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:27.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:27.687+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:27.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:27.833+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:27.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:27.833+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:27.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:27.833+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:27.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:27 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:27.833+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:28.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:27 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:27.553+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:28.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:27 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:27.553+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:28.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:27 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:27.553+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:28.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:27 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:27.553+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:28.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:27.946+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:28.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:27.946+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:28.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:27.946+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:28.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:27.946+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:28.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:27 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:27.946+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:28.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:28.611+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:28.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:28.611+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:28.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:28.611+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:28.865 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:28.611+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.691155+0000 front 2024-03-16T10:27:00.691128+0000 (oldest deadline 2024-03-16T10:27:25.391145+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:28.686+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:28.686+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:28.686+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:28.686+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:28.823+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:28.823+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:28.823+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:28.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:28 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:28.823+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:29.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:28.566+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:29.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:28.566+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:29.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:28.566+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:29.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:28 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:28.566+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:29.277 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:28.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:29.278 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:28.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:29.278 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:28.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:29.278 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:28.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:29.278 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:28 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:28.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:29.664+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:29.664+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:29.664+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:29.664+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:29.844+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:29.844+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:29.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:29.844+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:29.957 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:29 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:29.844+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:29.960 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:29.614+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.787463+0000 front 2024-03-16T10:26:33.787497+0000 (oldest deadline 2024-03-16T10:26:56.087581+0000) 2024-03-16T10:27:29.960 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:29.614+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:42.489202+0000 front 2024-03-16T10:26:42.488965+0000 (oldest deadline 2024-03-16T10:27:08.388926+0000) 2024-03-16T10:27:29.960 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:29.614+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.389784+0000 front 2024-03-16T10:26:48.389671+0000 (oldest deadline 2024-03-16T10:27:17.190067+0000) 2024-03-16T10:27:29.960 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7[54898]: 2024-03-16T10:27:29.614+0000 7f2a8d25f700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.691155+0000 front 2024-03-16T10:27:00.691128+0000 (oldest deadline 2024-03-16T10:27:25.391145+0000) 2024-03-16T10:27:30.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:29.549+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:30.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:29.549+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:30.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:29.549+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:30.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:29 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:29.549+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:30.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:29.959+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:30.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:29.959+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:30.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:29.959+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:30.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:29.959+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:30.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:29.959+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:30.867+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:30.867+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:30.867+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:30.867+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:30.678+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:30.678+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:30.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:30.678+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:30.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:30 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:30.678+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:31.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:30.510+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:31.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:30.510+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:31.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:30.510+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:31.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:30 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:30.510+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:31.364 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:31 smithi105 podman[74698]: 2024-03-16 10:27:31.125170519 +0000 UTC m=+5.938262796 container died 6f87be7c79cc35f26d2c7cc9e4a8abb521f103b233bb608e69cf5913e495ccbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2) 2024-03-16T10:27:31.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:30.975+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:31.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:30.975+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:31.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:30.975+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:31.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:30.975+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:31.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:30 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:30.975+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:31.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:31.483+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:31.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:31.483+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:31.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:31.483+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:31.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:31 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:31.483+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:31.794 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:31 smithi105 podman[74698]: 2024-03-16 10:27:31.533677342 +0000 UTC m=+6.346769618 container cleanup 6f87be7c79cc35f26d2c7cc9e4a8abb521f103b233bb608e69cf5913e495ccbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-16T10:27:31.794 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:31 smithi105 bash[74698]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:31.882+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:31.882+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:31.882+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:31.882+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:31.633+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:31.633+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:31.633+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:31.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:31 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:31.633+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:32.114 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:31 smithi105 podman[74718]: 2024-03-16 10:27:31.793664683 +0000 UTC m=+0.663549639 container remove 6f87be7c79cc35f26d2c7cc9e4a8abb521f103b233bb608e69cf5913e495ccbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:27:32.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:32.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:32.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:32.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:32.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:32.448 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.085304609 +0000 UTC m=+0.017824291 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:32.448 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.211511923 +0000 UTC m=+0.144031601 container create 14063086941f29c26aaf4f0b6bbc84dc454c18747740890ca485db7c13b6e923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:27:32.448 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.448149804 +0000 UTC m=+0.380669486 container init 14063086941f29c26aaf4f0b6bbc84dc454c18747740890ca485db7c13b6e923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T10:27:32.738 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.45147672 +0000 UTC m=+0.383996406 container start 14063086941f29c26aaf4f0b6bbc84dc454c18747740890ca485db7c13b6e923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True) 2024-03-16T10:27:32.738 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.573475202 +0000 UTC m=+0.505994889 container attach 14063086941f29c26aaf4f0b6bbc84dc454c18747740890ca485db7c13b6e923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T10:27:32.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:32.440+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:32.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:32.440+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:32.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:32.440+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:32.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:32 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:32.440+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:32.598+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:32.598+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:32.598+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:32.598+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:32.837+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:32.837+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:32.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:32.837+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:32.957 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:32 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:32.837+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:32.990 INFO:journalctl@ceph.osd.7.smithi105.stdout:Mar 16 10:27:32 smithi105 podman[74809]: 2024-03-16 10:27:32.737997138 +0000 UTC m=+0.670516810 container died 14063086941f29c26aaf4f0b6bbc84dc454c18747740890ca485db7c13b6e923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T10:27:33.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:33.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:33.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:33.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:33.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:33.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:32 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:32.989+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:33.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.7.service' 2024-03-16T10:27:33.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:33.457 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-16T10:27:33.457 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-16T10:27:33.457 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.8 2024-03-16T10:27:33.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:33.446+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:33.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:33.446+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:33.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:33.446+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:33.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:33 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:33.446+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 systemd[1]: Stopping Ceph osd.8 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.578+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.578+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.578+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.578+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.578+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.770+0000 7fc29d8e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.770+0000 7fc29d8e8700 -1 osd.8 121 *** Got signal Terminated *** 2024-03-16T10:27:33.772 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:33.770+0000 7fc29d8e8700 -1 osd.8 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:34.205 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:33.802+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:34.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:33.802+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:34.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:33.802+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:34.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:33 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:33.802+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:34.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:34.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:34.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:34.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:34.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:34.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:34.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:34.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:34.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:34.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:34.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:34.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:34.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:34 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:34.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:34.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:34.770+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:34.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:34.770+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:34.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:34.770+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:34.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:34.770+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:34.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:34.567+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:34.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:34.567+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:34.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:34.567+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:34.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:34.567+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:34.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:34 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:34.567+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:35.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:35.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:35.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:35.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:35.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:35.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:34 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:34.970+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:35.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:35 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:35.476+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:35.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:35 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:35.476+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:35.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:35 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:35.476+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:35.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:35 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:35.476+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:35.594+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:35.594+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:35.594+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:35.594+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:35.594+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:35.775+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:35.775+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:35.775+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:35.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:35 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:35.775+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:36.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:36.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:36.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:36.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:36.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:36.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:35 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:35.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:36.631+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:36.631+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:36.631+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:36.631+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:36.631+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:36.784+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:36.784+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:36.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:36.784+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:36.957 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:36.784+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:36.957 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:36 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:36.784+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:37.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:36.505+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:37.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:36.505+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:37.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:36.505+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:37.005 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:36 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:36.505+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:37.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:36 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:36.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:37.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:37.503+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:37.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:37.503+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:37.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:37.503+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:37.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:37.503+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:37.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:37 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:37.503+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:37.766+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:37.766+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:37.766+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:37.766+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:37.766+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:37.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:37.956 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:37.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:37.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:37.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:37.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:37.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:37.957 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:37 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:37.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:38.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:38.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:38.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:38.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:38.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:38.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:37.976+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:38.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:38 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:38.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:38.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:38 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:38.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:38.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:38 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:38.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:38.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:38 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:38.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:38.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:38 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:38.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:38.869 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:38.720+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:38.720+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:38.720+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:38.720+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:38.720+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:38.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:23.652752+0000 front 2024-03-16T10:26:23.652643+0000 (oldest deadline 2024-03-16T10:26:45.253123+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:38.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:31.456208+0000 front 2024-03-16T10:26:31.456448+0000 (oldest deadline 2024-03-16T10:27:00.157561+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:38.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:43.658588+0000 front 2024-03-16T10:26:43.658495+0000 (oldest deadline 2024-03-16T10:27:07.059512+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:38.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:02.262423+0000 front 2024-03-16T10:27:02.262554+0000 (oldest deadline 2024-03-16T10:27:28.062823+0000) 2024-03-16T10:27:38.870 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:38 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8[55416]: 2024-03-16T10:27:38.610+0000 7fc295ed4700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:08.063264+0000 front 2024-03-16T10:27:08.063352+0000 (oldest deadline 2024-03-16T10:27:32.763538+0000) 2024-03-16T10:27:39.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:39.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:39.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:39.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:39.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:39.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:38 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:38.974+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:39.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:39.456+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:39.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:39.456+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:39.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:39.456+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:39.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:39.456+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:39.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:39 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:39.456+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:39.905 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:39 smithi194 podman[75818]: 2024-03-16 10:27:39.618286653 +0000 UTC m=+5.964811334 container died ae689398ce1b424b9242f0815cf6353270741506b17929252842941c6b043928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:27:39.905 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:39.679+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:39.905 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:39.679+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:39.905 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:39.679+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:39.906 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:39.679+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:39.906 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:39 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:39.679+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:40.207 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75818]: 2024-03-16 10:27:40.00462647 +0000 UTC m=+6.351151142 container cleanup ae689398ce1b424b9242f0815cf6353270741506b17929252842941c6b043928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:27:40.207 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 bash[75818]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8 2024-03-16T10:27:40.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:40.366 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:40.366 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:40.366 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:40.366 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:40.366 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:40 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:40.009+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:40.641 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75846]: 2024-03-16 10:27:40.298039322 +0000 UTC m=+0.676679809 container remove ae689398ce1b424b9242f0815cf6353270741506b17929252842941c6b043928 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-16T10:27:40.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:40 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:40.458+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:40.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:40 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:40.458+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:40.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:40 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:40.458+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:40.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:40 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:40.458+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:40.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:40 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:40.458+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:40.926 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75937]: 2024-03-16 10:27:40.559309244 +0000 UTC m=+0.017980671 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75937]: 2024-03-16 10:27:40.665336244 +0000 UTC m=+0.124007659 container create dd743f9eaa0635175a88595670c59c85726eb32c647125ad5966192c8f9824bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75937]: 2024-03-16 10:27:40.926128239 +0000 UTC m=+0.384799646 container init dd743f9eaa0635175a88595670c59c85726eb32c647125ad5966192c8f9824bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:40.641+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:40.641+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:40.641+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:40.641+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:40.927 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:40 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:40.641+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:41.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:40 smithi194 podman[75937]: 2024-03-16 10:27:40.92994294 +0000 UTC m=+0.388614359 container start dd743f9eaa0635175a88595670c59c85726eb32c647125ad5966192c8f9824bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-16T10:27:41.206 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:41 smithi194 podman[75937]: 2024-03-16 10:27:41.034749573 +0000 UTC m=+0.493420991 container attach dd743f9eaa0635175a88595670c59c85726eb32c647125ad5966192c8f9824bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T10:27:41.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:41.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:41.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:41.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:41.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:41.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:41 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:41.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:41.512 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 10:27:41 smithi194 podman[75937]: 2024-03-16 10:27:41.215958205 +0000 UTC m=+0.674629620 container died dd743f9eaa0635175a88595670c59c85726eb32c647125ad5966192c8f9824bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T10:27:41.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:41 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:41.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:41.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:41 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:41.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:41.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:41 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:41.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:41.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:41 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:41.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:41.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:41 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:41.475+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:41.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:41.676+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:41.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:41.676+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:41.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:41.676+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:41.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:41.676+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:41.830 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:41 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:41.676+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:41.848 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.8.service' 2024-03-16T10:27:41.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:41.893 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-16T10:27:41.893 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-16T10:27:41.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.9 2024-03-16T10:27:42.255 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 systemd[1]: Stopping Ceph osd.9 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:42.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:42.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:42.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:42.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:42.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:42.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:42 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:42.051+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:42.500 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.478+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:42.500 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.478+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:42.500 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.478+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:42.500 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.478+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:42.500 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.478+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:42.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.523+0000 7f729de00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:42.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.523+0000 7f729de00700 -1 osd.9 121 *** Got signal Terminated *** 2024-03-16T10:27:42.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:42 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:42.523+0000 7f729de00700 -1 osd.9 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:42.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:42.686+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:42.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:42.686+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:42.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:42.686+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:42.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:42.686+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:42.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:42.686+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:43.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:43.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:43 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:43.017+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:43.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:43.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:43.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:43.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:43.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:43.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:43 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:43.462+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:16.881592+0000 front 2024-03-16T10:27:16.881832+0000 (oldest deadline 2024-03-16T10:27:42.781513+0000) 2024-03-16T10:27:43.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:43 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:43.704+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:43.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:43 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:43.704+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:43.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:43 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:43.704+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:43.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:43 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:43.704+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:43.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:43 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:43.704+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:44.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:44.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:44 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:44.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:44.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:44.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:44.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:44.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:44.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:44.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:44 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:44.426+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:16.881592+0000 front 2024-03-16T10:27:16.881832+0000 (oldest deadline 2024-03-16T10:27:42.781513+0000) 2024-03-16T10:27:44.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:44 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:44.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:44.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:44 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:44.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:44.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:44 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:44.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:44.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:44 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:44.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:44.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:44 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:44.702+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:45.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:45.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:45 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:45.006+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:45.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:45 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:45.401+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:16.881592+0000 front 2024-03-16T10:27:16.881832+0000 (oldest deadline 2024-03-16T10:27:42.781513+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:45.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:45.661+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:46.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:46.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:46 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:46.015+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:46.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:46.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:46.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:46.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:46.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:46.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:46 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:46.431+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:16.881592+0000 front 2024-03-16T10:27:16.881832+0000 (oldest deadline 2024-03-16T10:27:42.781513+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:46.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:46 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:46.637+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:47.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:47.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:47 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:47.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:47.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:20.166206+0000 front 2024-03-16T10:26:20.166553+0000 (oldest deadline 2024-03-16T10:26:49.567602+0000) 2024-03-16T10:27:47.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:29.568871+0000 front 2024-03-16T10:26:29.568590+0000 (oldest deadline 2024-03-16T10:26:55.969728+0000) 2024-03-16T10:27:47.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.673292+0000 front 2024-03-16T10:26:41.673427+0000 (oldest deadline 2024-03-16T10:27:04.573499+0000) 2024-03-16T10:27:47.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:48.576954+0000 front 2024-03-16T10:26:48.577093+0000 (oldest deadline 2024-03-16T10:27:19.777607+0000) 2024-03-16T10:27:47.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:10.980797+0000 front 2024-03-16T10:27:10.981073+0000 (oldest deadline 2024-03-16T10:27:36.880604+0000) 2024-03-16T10:27:47.756 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:47 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9[66773]: 2024-03-16T10:27:47.409+0000 7f7299600700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:16.881592+0000 front 2024-03-16T10:27:16.881832+0000 (oldest deadline 2024-03-16T10:27:42.781513+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:47.956 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:47 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:47.678+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:48.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:48 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:48.049+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:49.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:48 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:48.726+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:49.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:49.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:49 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:49.089+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:49.754 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:49 smithi039 podman[94506]: 2024-03-16 10:27:49.483015839 +0000 UTC m=+7.113855145 container died bf04eb387df55ae6c875d378e9ffa8f023a4fe705178dd7805cffa2013c19185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:27:50.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:50.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:50.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:50.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:50.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:50.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:49.752+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:50.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:50 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:50.056+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:50.506 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:50 smithi039 podman[94506]: 2024-03-16 10:27:50.132136582 +0000 UTC m=+7.762975875 container cleanup bf04eb387df55ae6c875d378e9ffa8f023a4fe705178dd7805cffa2013c19185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307) 2024-03-16T10:27:50.506 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:50 smithi039 bash[94506]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9 2024-03-16T10:27:50.506 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:50 smithi039 podman[94525]: 2024-03-16 10:27:50.324889597 +0000 UTC m=+0.817283448 container remove bf04eb387df55ae6c875d378e9ffa8f023a4fe705178dd7805cffa2013c19185 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:51.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:50 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:50.765+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:51.330 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:50.98270789 +0000 UTC m=+0.034652716 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:27:51.330 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:51.084162577 +0000 UTC m=+0.136107439 container create a812828f56f9da4b1b90b7f7b215ede323602d6a65c20102acbb2f6451e8d73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-16T10:27:51.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:51.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:51 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:51.045+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:51.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:51.32964026 +0000 UTC m=+0.381585125 container init a812828f56f9da4b1b90b7f7b215ede323602d6a65c20102acbb2f6451e8d73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T10:27:51.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:51.340966492 +0000 UTC m=+0.392911378 container start a812828f56f9da4b1b90b7f7b215ede323602d6a65c20102acbb2f6451e8d73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T10:27:51.755 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:51.428465668 +0000 UTC m=+0.480410567 container attach a812828f56f9da4b1b90b7f7b215ede323602d6a65c20102acbb2f6451e8d73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-16T10:27:52.136 INFO:journalctl@ceph.osd.9.smithi039.stdout:Mar 16 10:27:51 smithi039 podman[94622]: 2024-03-16 10:27:51.784948 +0000 UTC m=+0.836892882 container died a812828f56f9da4b1b90b7f7b215ede323602d6a65c20102acbb2f6451e8d73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:52.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:51 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:51.776+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:52.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:52.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:52.057+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:52.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.9.service' 2024-03-16T10:27:52.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:27:52.799 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-16T10:27:52.799 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-16T10:27:52.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.10 2024-03-16T10:27:53.114 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:52 smithi105 systemd[1]: Stopping Ceph osd.10 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:53.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.034+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:53.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:52 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:52.730+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:53.614 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.149+0000 7f3486dc7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:27:53.615 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.149+0000 7f3486dc7700 -1 osd.10 121 *** Got signal Terminated *** 2024-03-16T10:27:53.615 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:53 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:53.149+0000 7f3486dc7700 -1 osd.10 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:54.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:53.729+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:54.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.001+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:55.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:54 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:54.751+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:55.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:54 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:54.958+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:56.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:56.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:55 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:55.789+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:27:56.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:56.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:55 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:55.994+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:57.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:57.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:57.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:57.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:57.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:57.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:57.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:56 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:56.839+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:27:57.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:57.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:56 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:56.952+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:58.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:57 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:57.803+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.787051+0000 front 2024-03-16T10:26:22.786834+0000 (oldest deadline 2024-03-16T10:26:47.987335+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:33.188833+0000 front 2024-03-16T10:26:33.188867+0000 (oldest deadline 2024-03-16T10:26:59.088966+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:41.990338+0000 front 2024-03-16T10:26:41.990326+0000 (oldest deadline 2024-03-16T10:27:05.990549+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-03-16T10:26:49.392207+0000 front 2024-03-16T10:26:49.392644+0000 (oldest deadline 2024-03-16T10:27:14.692044+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:00.593071+0000 front 2024-03-16T10:27:00.593035+0000 (oldest deadline 2024-03-16T10:27:24.693097+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:11.094712+0000 front 2024-03-16T10:27:11.094648+0000 (oldest deadline 2024-03-16T10:27:32.794565+0000) 2024-03-16T10:27:58.365 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:57 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10[61666]: 2024-03-16T10:27:57.963+0000 7f34833bb700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.796126+0000 front 2024-03-16T10:27:19.795975+0000 (oldest deadline 2024-03-16T10:27:42.095827+0000) 2024-03-16T10:27:59.089 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:58 smithi105 podman[74937]: 2024-03-16 10:27:58.831301909 +0000 UTC m=+5.825753840 container died 4a06c07ca4f984e10ed9005e066d75f1b09bf85b778d5e0f2378d6c1f55be611 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:27:59.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:58 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:58.847+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:27:59.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:59 smithi105 podman[74937]: 2024-03-16 10:27:59.174069841 +0000 UTC m=+6.168521762 container cleanup 4a06c07ca4f984e10ed9005e066d75f1b09bf85b778d5e0f2378d6c1f55be611 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T10:27:59.364 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:59 smithi105 bash[74937]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10 2024-03-16T10:27:59.837 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:59 smithi105 podman[74954]: 2024-03-16 10:27:59.4307676 +0000 UTC m=+0.595524449 container remove 4a06c07ca4f984e10ed9005e066d75f1b09bf85b778d5e0f2378d6c1f55be611 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:28:00.114 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:59 smithi105 podman[75048]: 2024-03-16 10:27:59.740456731 +0000 UTC m=+0.017697830 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:28:00.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:27:59 smithi105 podman[75048]: 2024-03-16 10:27:59.84821161 +0000 UTC m=+0.125452707 container create 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T10:28:00.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 podman[75048]: 2024-03-16 10:28:00.042830412 +0000 UTC m=+0.320071510 container init 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T10:28:00.115 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 podman[75048]: 2024-03-16 10:28:00.046182395 +0000 UTC m=+0.323423485 container start 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:00.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:27:59 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:27:59.828+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:00.615 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 podman[75048]: 2024-03-16 10:28:00.117231726 +0000 UTC m=+0.394472840 container attach 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T10:28:00.615 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 podman[75048]: 2024-03-16 10:28:00.331694032 +0000 UTC m=+0.608935136 container died 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:28:00.861 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.10.service' 2024-03-16T10:28:00.887 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 podman[75074]: 2024-03-16 10:28:00.845478887 +0000 UTC m=+0.504158169 container remove 3203b0991153001554931331411011b2733b18c18646eb8c01b429b0ca89a0bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-10-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T10:28:00.887 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-16T10:28:00.887 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 systemd[1]: Stopped Ceph osd.10 for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:28:00.887 INFO:journalctl@ceph.osd.10.smithi105.stdout:Mar 16 10:28:00 smithi105 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.10.service: Consumed 9min 45.975s CPU time. 2024-03-16T10:28:00.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:00.902 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-16T10:28:00.902 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-16T10:28:00.902 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.11 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:00.824+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:01.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:00 smithi194 systemd[1]: Stopping Ceph osd.11 for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:01.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.223+0000 7f9d4077e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:01.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.223+0000 7f9d4077e700 -1 osd.11 121 *** Got signal Terminated *** 2024-03-16T10:28:01.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.223+0000 7f9d4077e700 -1 osd.11 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:02.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:01 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:01.873+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:03.205 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:03.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:02 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:02.889+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:04.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:03 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:03.859+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:05.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:04 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:04.856+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-03-16T10:26:22.057425+0000 front 2024-03-16T10:26:22.057247+0000 (oldest deadline 2024-03-16T10:26:46.657734+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6804 osd.1 since back 2024-03-16T10:26:34.259602+0000 front 2024-03-16T10:26:34.259390+0000 (oldest deadline 2024-03-16T10:26:55.959552+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T10:26:40.660886+0000 front 2024-03-16T10:26:40.660870+0000 (oldest deadline 2024-03-16T10:27:09.461431+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6812 osd.4 since back 2024-03-16T10:27:01.763503+0000 front 2024-03-16T10:27:01.763481+0000 (oldest deadline 2024-03-16T10:27:27.663547+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T10:27:09.964568+0000 front 2024-03-16T10:27:09.964630+0000 (oldest deadline 2024-03-16T10:27:35.864740+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-03-16T10:27:19.965872+0000 front 2024-03-16T10:27:19.965762+0000 (oldest deadline 2024-03-16T10:27:45.265770+0000) 2024-03-16T10:28:06.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.840+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.105:6820 osd.7 since back 2024-03-16T10:27:29.367012+0000 front 2024-03-16T10:27:29.367028+0000 (oldest deadline 2024-03-16T10:27:54.666983+0000) 2024-03-16T10:28:06.207 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:05 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11[62372]: 2024-03-16T10:28:05.841+0000 7f9d3cd72700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T10:27:36.367847+0000 front 2024-03-16T10:27:36.367810+0000 (oldest deadline 2024-03-16T10:27:59.867859+0000) 2024-03-16T10:28:07.281 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:06 smithi194 podman[76042]: 2024-03-16 10:28:06.996832928 +0000 UTC m=+5.897759313 container died 27f95af5f0de9aaa27e0688ccee625da60cc6b54401d22a45e2fb2b6253217a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:28:07.532 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:07 smithi194 podman[76042]: 2024-03-16 10:28:07.347472993 +0000 UTC m=+6.248399366 container cleanup 27f95af5f0de9aaa27e0688ccee625da60cc6b54401d22a45e2fb2b6253217a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T10:28:07.532 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:07 smithi194 bash[76042]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11 2024-03-16T10:28:07.940 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:07 smithi194 podman[76059]: 2024-03-16 10:28:07.531879881 +0000 UTC m=+0.520787147 container remove 27f95af5f0de9aaa27e0688ccee625da60cc6b54401d22a45e2fb2b6253217a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T10:28:08.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:07 smithi194 podman[76151]: 2024-03-16 10:28:07.94088429 +0000 UTC m=+0.106613866 container create 2ea7a3c7f8892ed80780b38fc129305d5a26b599e7f10848d5730b706047f2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T10:28:08.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:07 smithi194 podman[76151]: 2024-03-16 10:28:07.851752029 +0000 UTC m=+0.017481615 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:28:08.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:08 smithi194 podman[76151]: 2024-03-16 10:28:08.128568215 +0000 UTC m=+0.294297791 container init 2ea7a3c7f8892ed80780b38fc129305d5a26b599e7f10848d5730b706047f2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T10:28:08.206 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:08 smithi194 podman[76151]: 2024-03-16 10:28:08.131863271 +0000 UTC m=+0.297592848 container start 2ea7a3c7f8892ed80780b38fc129305d5a26b599e7f10848d5730b706047f2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-16T10:28:08.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:08 smithi194 podman[76151]: 2024-03-16 10:28:08.211764667 +0000 UTC m=+0.377494253 container attach 2ea7a3c7f8892ed80780b38fc129305d5a26b599e7f10848d5730b706047f2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T10:28:08.706 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 10:28:08 smithi194 podman[76151]: 2024-03-16 10:28:08.418310737 +0000 UTC m=+0.584040308 container died 2ea7a3c7f8892ed80780b38fc129305d5a26b599e7f10848d5730b706047f2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-16T10:28:08.983 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@osd.11.service' 2024-03-16T10:28:09.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:09.024 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-16T10:28:09.025 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-16T10:28:09.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.a 2024-03-16T10:28:09.505 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:09 smithi039 systemd[1]: Stopping Ceph mds.a for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:10.005 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: 2024-03-16T10:28:09.614+0000 7fa669a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:10.005 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:09 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a[73578]: 2024-03-16T10:28:09.614+0000 7fa669a00700 -1 mds.a *** got signal Terminated *** 2024-03-16T10:28:12.506 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:12 smithi039 podman[94727]: 2024-03-16 10:28:12.224840088 +0000 UTC m=+2.743995621 container died fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:28:12.926 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:12 smithi039 podman[94727]: 2024-03-16 10:28:12.641008948 +0000 UTC m=+3.160164487 container cleanup fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-16T10:28:12.926 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:12 smithi039 bash[94727]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a 2024-03-16T10:28:13.254 INFO:journalctl@ceph.mds.a.smithi039.stdout:Mar 16 10:28:12 smithi039 podman[94741]: 2024-03-16 10:28:12.925178678 +0000 UTC m=+0.676450714 container remove fb3123655dec646a4c8695e9441c6d85296747112fea3955bb2d9a402ca008be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:28:13.469 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.a.service' 2024-03-16T10:28:13.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:13.591 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-16T10:28:13.592 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-16T10:28:13.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.d 2024-03-16T10:28:14.005 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:13 smithi039 systemd[1]: Stopping Ceph mds.d for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:14.505 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: 2024-03-16T10:28:14.218+0000 7f2ca3400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:14.505 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:14 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d[72594]: 2024-03-16T10:28:14.218+0000 7f2ca3400700 -1 mds.d *** got signal Terminated *** 2024-03-16T10:28:16.505 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:16 smithi039 podman[94846]: 2024-03-16 10:28:16.220594063 +0000 UTC m=+2.128576596 container died f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-16T10:28:16.954 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:16 smithi039 podman[94846]: 2024-03-16 10:28:16.777867883 +0000 UTC m=+2.685850409 container cleanup f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T10:28:16.955 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:16 smithi039 bash[94846]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d 2024-03-16T10:28:17.254 INFO:journalctl@ceph.mds.d.smithi039.stdout:Mar 16 10:28:16 smithi039 podman[94860]: 2024-03-16 10:28:16.953787057 +0000 UTC m=+0.722037067 container remove f06753bec5ddca57174c7473239ac0ecf55df25beeeccc12df731631c955c11a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:28:17.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.d.service' 2024-03-16T10:28:17.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:17.627 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-16T10:28:17.627 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-16T10:28:17.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.g 2024-03-16T10:28:18.005 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:17 smithi039 systemd[1]: Stopping Ceph mds.g for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:18.505 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g[73230]: 2024-03-16T10:28:18.246+0000 7f42b0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:18.505 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:18 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g[73230]: 2024-03-16T10:28:18.246+0000 7f42b0c00700 -1 mds.g *** got signal Terminated *** 2024-03-16T10:28:20.377 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:20 smithi039 podman[94966]: 2024-03-16 10:28:20.124753636 +0000 UTC m=+2.006048148 container died 61a51e22522b0899f97643d625ddcce9469a61a68fb023f214f9a57969bc1115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:28:20.738 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:20 smithi039 podman[94966]: 2024-03-16 10:28:20.522837294 +0000 UTC m=+2.404131799 container cleanup 61a51e22522b0899f97643d625ddcce9469a61a68fb023f214f9a57969bc1115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:28:20.738 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:20 smithi039 bash[94966]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g 2024-03-16T10:28:21.005 INFO:journalctl@ceph.mds.g.smithi039.stdout:Mar 16 10:28:20 smithi039 podman[94980]: 2024-03-16 10:28:20.890776522 +0000 UTC m=+0.731089446 container remove 61a51e22522b0899f97643d625ddcce9469a61a68fb023f214f9a57969bc1115 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-16T10:28:21.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.g.service' 2024-03-16T10:28:21.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:21.472 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-16T10:28:21.472 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-16T10:28:21.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.j 2024-03-16T10:28:22.005 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:21 smithi039 systemd[1]: Stopping Ceph mds.j for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:22.505 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: 2024-03-16T10:28:22.057+0000 7f2943a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:22.505 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:22 smithi039 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j[72244]: 2024-03-16T10:28:22.057+0000 7f2943a00700 -1 mds.j *** got signal Terminated *** 2024-03-16T10:28:24.505 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:24 smithi039 podman[95094]: 2024-03-16 10:28:24.243951649 +0000 UTC m=+2.327059881 container died e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:28:24.931 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:24 smithi039 podman[95094]: 2024-03-16 10:28:24.737815169 +0000 UTC m=+2.820923409 container cleanup e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-16T10:28:24.931 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:24 smithi039 bash[95094]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j 2024-03-16T10:28:25.255 INFO:journalctl@ceph.mds.j.smithi039.stdout:Mar 16 10:28:24 smithi039 podman[95109]: 2024-03-16 10:28:24.930163327 +0000 UTC m=+0.671862624 container remove e85b2522ff1fb0143b6582ff117a25355ff2589845dd233c41c9da529b8aad7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T10:28:25.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.j.service' 2024-03-16T10:28:25.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:25.585 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-16T10:28:25.585 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-16T10:28:25.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.b 2024-03-16T10:28:25.880 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:25 smithi105 systemd[1]: Stopping Ceph mds.b for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:26.364 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:28:25.878+0000 7fbaa283b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:26.365 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:25 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b[67214]: 2024-03-16T10:28:25.878+0000 7fbaa283b700 -1 mds.b *** got signal Terminated *** 2024-03-16T10:28:28.439 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:28 smithi105 podman[75154]: 2024-03-16 10:28:28.146640754 +0000 UTC m=+2.366449732 container died c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-16T10:28:28.806 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:28 smithi105 podman[75154]: 2024-03-16 10:28:28.521207076 +0000 UTC m=+2.741016038 container cleanup c62587e2ba24122045c1801970fd7cab12b7e32b2e788c1f2ba9a32dacc319b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:28:28.806 INFO:journalctl@ceph.mds.b.smithi105.stdout:Mar 16 10:28:28 smithi105 bash[75154]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-b 2024-03-16T10:28:28.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.b.service' 2024-03-16T10:28:28.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:28.946 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-16T10:28:28.946 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-16T10:28:28.946 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.e 2024-03-16T10:28:29.114 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:29 smithi105 systemd[1]: Stopping Ceph mds.e for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:29.615 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: 2024-03-16T10:28:29.257+0000 7f90744d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:29.615 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:29 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e[66590]: 2024-03-16T10:28:29.257+0000 7f90744d3700 -1 mds.e *** got signal Terminated *** 2024-03-16T10:28:32.414 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:32 smithi105 podman[75286]: 2024-03-16 10:28:32.160966801 +0000 UTC m=+3.000875172 container died 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-16T10:28:32.701 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:32 smithi105 podman[75286]: 2024-03-16 10:28:32.491197384 +0000 UTC m=+3.331105733 container cleanup 1dd69e54c455e708d6dace59d0dbfc130680e43e4b15a66b156e89d219443319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T10:28:32.701 INFO:journalctl@ceph.mds.e.smithi105.stdout:Mar 16 10:28:32 smithi105 bash[75286]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-e 2024-03-16T10:28:32.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.e.service' 2024-03-16T10:28:32.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:32.952 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-16T10:28:32.952 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-16T10:28:32.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.h 2024-03-16T10:28:33.278 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:33 smithi105 systemd[1]: Stopping Ceph mds.h for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:33.614 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:33 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h[66902]: 2024-03-16T10:28:33.277+0000 7fb8eedb0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:33.614 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:33 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h[66902]: 2024-03-16T10:28:33.277+0000 7fb8eedb0700 -1 mds.h *** got signal Terminated *** 2024-03-16T10:28:36.422 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:36 smithi105 podman[75407]: 2024-03-16 10:28:36.156863133 +0000 UTC m=+2.989008963 container died 4bddcc731235f081e962a03ba6c9317dd05ea43384f5bad26aed22aabbc9f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-16T10:28:36.787 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:36 smithi105 podman[75407]: 2024-03-16 10:28:36.511470675 +0000 UTC m=+3.343616496 container cleanup 4bddcc731235f081e962a03ba6c9317dd05ea43384f5bad26aed22aabbc9f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T10:28:36.788 INFO:journalctl@ceph.mds.h.smithi105.stdout:Mar 16 10:28:36 smithi105 bash[75407]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-h 2024-03-16T10:28:36.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.h.service' 2024-03-16T10:28:36.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:36.957 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-16T10:28:36.957 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-16T10:28:36.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.k 2024-03-16T10:28:37.365 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:37 smithi105 systemd[1]: Stopping Ceph mds.k for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:37.365 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k[66277]: 2024-03-16T10:28:37.272+0000 7f4e42d86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:37.365 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:37 smithi105 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k[66277]: 2024-03-16T10:28:37.272+0000 7f4e42d86700 -1 mds.k *** got signal Terminated *** 2024-03-16T10:28:41.554 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:41 smithi105 podman[75529]: 2024-03-16 10:28:41.30031777 +0000 UTC m=+4.121730600 container died a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:28:41.816 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:41 smithi105 podman[75529]: 2024-03-16 10:28:41.631794157 +0000 UTC m=+4.453207030 container cleanup a18fcad48ee7b1c1fe0876d76893df31c888e2ba669706233c5e9f3f1b150edb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T10:28:41.816 INFO:journalctl@ceph.mds.k.smithi105.stdout:Mar 16 10:28:41 smithi105 bash[75529]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-k 2024-03-16T10:28:42.036 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.k.service' 2024-03-16T10:28:42.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:42.071 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-16T10:28:42.071 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-16T10:28:42.071 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.c 2024-03-16T10:28:42.364 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:42 smithi194 systemd[1]: Stopping Ceph mds.c for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:42.705 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: 2024-03-16T10:28:42.362+0000 7f9e94bd2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:42.706 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:42 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c[67676]: 2024-03-16T10:28:42.362+0000 7f9e94bd2700 -1 mds.c *** got signal Terminated *** 2024-03-16T10:28:44.705 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:44 smithi194 podman[76258]: 2024-03-16 10:28:44.393070822 +0000 UTC m=+2.128509333 container died a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-16T10:28:44.705 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:44 smithi194 podman[76258]: 2024-03-16 10:28:44.704940651 +0000 UTC m=+2.440379142 container cleanup a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T10:28:44.956 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:44 smithi194 bash[76258]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c 2024-03-16T10:28:44.956 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 10:28:44 smithi194 podman[76283]: 2024-03-16 10:28:44.931340291 +0000 UTC m=+0.537884246 container remove a01a0d3ac689917e3ec6cb710b45805964133150eede3498a99ee77beac8d8fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-c, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:28:45.172 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.c.service' 2024-03-16T10:28:45.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:45.211 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-16T10:28:45.211 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-16T10:28:45.211 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.f 2024-03-16T10:28:45.457 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:45 smithi194 systemd[1]: Stopping Ceph mds.f for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:45.956 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f[67372]: 2024-03-16T10:28:45.558+0000 7f9efaf0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:45.956 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:45 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f[67372]: 2024-03-16T10:28:45.558+0000 7f9efaf0c700 -1 mds.f *** got signal Terminated *** 2024-03-16T10:28:48.640 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:48 smithi194 podman[76391]: 2024-03-16 10:28:48.386018347 +0000 UTC m=+2.957681918 container died 2b39b6917cfb67f70ff49eb3806e52766a76ac22430e849c184829e77cfc4f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T10:28:48.918 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:48 smithi194 podman[76391]: 2024-03-16 10:28:48.725050245 +0000 UTC m=+3.296713844 container cleanup 2b39b6917cfb67f70ff49eb3806e52766a76ac22430e849c184829e77cfc4f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-16T10:28:48.918 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:48 smithi194 bash[76391]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f 2024-03-16T10:28:49.150 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.f.service' 2024-03-16T10:28:49.174 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:48 smithi194 podman[76405]: 2024-03-16 10:28:48.917698219 +0000 UTC m=+0.530059735 container remove 2b39b6917cfb67f70ff49eb3806e52766a76ac22430e849c184829e77cfc4f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:28:49.174 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:49 smithi194 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-16T10:28:49.174 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:49 smithi194 systemd[1]: Stopped Ceph mds.f for 65a28618-e37b-11ee-95c9-87774f69a715. 2024-03-16T10:28:49.174 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 10:28:49 smithi194 systemd[1]: ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.f.service: Consumed 3.626s CPU time. 2024-03-16T10:28:49.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:49.186 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-16T10:28:49.187 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-16T10:28:49.187 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.i 2024-03-16T10:28:49.455 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:49 smithi194 systemd[1]: Stopping Ceph mds.i for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:49.955 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: 2024-03-16T10:28:49.483+0000 7fd900b36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:49.956 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:49 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i[67983]: 2024-03-16T10:28:49.483+0000 7fd900b36700 -1 mds.i *** got signal Terminated *** 2024-03-16T10:28:52.705 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:52 smithi194 podman[76512]: 2024-03-16 10:28:52.37151256 +0000 UTC m=+2.972873324 container died 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:28:53.021 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:52 smithi194 podman[76512]: 2024-03-16 10:28:52.802935628 +0000 UTC m=+3.404296340 container cleanup 5c5e101e3d6347e39c791914e9ef2da64cfbed1e979af9c850774fd4ecf7ea61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:28:53.021 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 10:28:52 smithi194 bash[76512]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-i 2024-03-16T10:28:53.151 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.i.service' 2024-03-16T10:28:53.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:53.187 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-16T10:28:53.187 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-16T10:28:53.187 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.l 2024-03-16T10:28:53.455 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:53 smithi194 systemd[1]: Stopping Ceph mds.l for 65a28618-e37b-11ee-95c9-87774f69a715... 2024-03-16T10:28:53.955 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: 2024-03-16T10:28:53.505+0000 7f0783dbd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:28:53.956 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:53 smithi194 ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l[67063]: 2024-03-16T10:28:53.506+0000 7f0783dbd700 -1 mds.l *** got signal Terminated *** 2024-03-16T10:28:56.706 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:56 smithi194 podman[76633]: 2024-03-16 10:28:56.369878519 +0000 UTC m=+2.967103187 container died 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-16T10:28:57.020 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:56 smithi194 podman[76633]: 2024-03-16 10:28:56.814573005 +0000 UTC m=+3.411797666 container cleanup 8e89948e3a38e0367263dc81c42bc3cf300b76b14e7f0a2b922a45f81999d92b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-16T10:28:57.021 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 10:28:56 smithi194 bash[76633]: ceph-65a28618-e37b-11ee-95c9-87774f69a715-mds-l 2024-03-16T10:28:57.216 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65a28618-e37b-11ee-95c9-87774f69a715@mds.l.service' 2024-03-16T10:28:57.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T10:28:57.250 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-16T10:28:57.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T10:28:57.595 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:29:02.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T10:29:02.699 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:29:05.445 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T10:29:05.645 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:29:08.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:29:08.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:29:08.340 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:29:08.369 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-16T10:29:08.371 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi039/crash 2024-03-16T10:29:08.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash -- . 2024-03-16T10:29:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T10:29:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-03-16T10:29:08.493 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi105/crash 2024-03-16T10:29:08.494 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash -- . 2024-03-16T10:29:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T10:29:08.545 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-03-16T10:29:08.547 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi194/crash 2024-03-16T10:29:08.548 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash -- . 2024-03-16T10:29:08.597 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/65a28618-e37b-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T10:29:08.597 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-03-16T10:29:08.598 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-16T10:29:08.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T10:29:08.782 INFO:teuthology.orchestra.run.smithi039.stdout:2024-03-16T09:59:20.780305+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T10:29:08.784 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-16T10:29:08.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[SEC\]' /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T10:29:08.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]' /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T10:29:08.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[WRN\]' /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T10:29:09.063 INFO:teuthology.orchestra.run.smithi039.stdout:2024-03-16T09:59:20.780305+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T10:29:09.064 INFO:tasks.cephadm:Compressing logs... 2024-03-16T10:29:09.064 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T10:29:09.068 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T10:29:09.071 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.l.log: /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log 2024-03-16T10:29:09.118 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-16T10:29:09.119 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-16T10:29:09.119 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T10:29:09.119 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-16T10:29:09.119 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-16T10:29:09.120 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-16T10:29:09.123 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-16T10:29:09.123 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.b.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-16T10:29:09.124 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.c.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-16T10:29:09.127 INFO:teuthology.orchestra.run.smithi105.stderr: 92.1% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-16T10:29:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-16T10:29:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:: No such file or directory 2024-03-16T10:29:09.167 INFO:teuthology.orchestra.run.smithi105.stderr: 83.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T10:29:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-16T10:29:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T10:29:09.174 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-16T10:29:09.175 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-16T10:29:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-16T10:29:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-16T10:29:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-16T10:29:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T10:29:09.180 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log 2024-03-16T10:29:09.181 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T10:29:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-16T10:29:09.183 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-16T10:29:09.183 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-16T10:29:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-16T10:29:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T10:29:09.186 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-16T10:29:09.186 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T10:29:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-16T10:29:09.242 INFO:teuthology.orchestra.run.smithi105.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T10:29:09.272 INFO:teuthology.orchestra.run.smithi039.stderr: 83.9% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T10:29:09.334 INFO:teuthology.orchestra.run.smithi039.stderr: 90.2% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T10:29:09.340 INFO:teuthology.orchestra.run.smithi105.stderr: 90.3% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T10:29:09.342 INFO:teuthology.orchestra.run.smithi105.stderr: 86.8% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T10:29:09.391 INFO:teuthology.orchestra.run.smithi194.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T10:29:09.393 INFO:teuthology.orchestra.run.smithi194.stderr: 83.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T10:29:09.394 INFO:teuthology.orchestra.run.smithi194.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T10:29:09.408 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T10:29:09.427 INFO:teuthology.orchestra.run.smithi039.stderr: 90.8% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T10:29:09.490 INFO:teuthology.orchestra.run.smithi039.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T10:29:09.548 INFO:teuthology.orchestra.run.smithi194.stderr: 86.8% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T10:29:09.673 INFO:teuthology.orchestra.run.smithi194.stderr: 90.9% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T10:29:09.711 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-16T10:29:09.900 INFO:teuthology.orchestra.run.smithi105.stderr: 93.7% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-16T10:29:10.373 INFO:teuthology.orchestra.run.smithi039.stderr: 93.2% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-16T10:29:10.533 INFO:teuthology.orchestra.run.smithi039.stderr: 93.1% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-16T10:29:11.370 INFO:teuthology.orchestra.run.smithi039.stderr: 90.1% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-16T10:29:11.841 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-16T10:29:12.290 INFO:teuthology.orchestra.run.smithi194.stderr: 94.1% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-16T10:29:13.374 INFO:teuthology.orchestra.run.smithi194.stderr: 91.8% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-16T10:29:19.142 INFO:teuthology.orchestra.run.smithi105.stderr: 91.6% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-16T10:29:19.665 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-16T10:29:26.215 INFO:teuthology.orchestra.run.smithi039.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-16T10:29:29.143 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-16T10:29:31.205 INFO:teuthology.orchestra.run.smithi039.stderr: 93.6% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-16T10:29:32.470 INFO:teuthology.orchestra.run.smithi194.stderr: 93.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-16T10:33:53.364 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-16T10:34:16.349 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-16T10:36:29.856 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-16T10:36:45.016 INFO:teuthology.orchestra.run.smithi194.stderr: 90.5% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-16T10:36:56.224 INFO:teuthology.orchestra.run.smithi105.stderr: 90.7% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-16T10:36:56.227 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-16T10:36:56.227 INFO:teuthology.orchestra.run.smithi105.stderr:real 7m47.142s 2024-03-16T10:36:56.227 INFO:teuthology.orchestra.run.smithi105.stderr:user 5m34.332s 2024-03-16T10:36:56.228 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m12.945s 2024-03-16T10:37:20.398 INFO:teuthology.orchestra.run.smithi194.stderr: 90.4% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-16T10:39:44.101 INFO:teuthology.orchestra.run.smithi194.stderr: 90.7% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-16T10:40:09.409 INFO:teuthology.orchestra.run.smithi194.stderr: 90.5% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-16T10:40:09.424 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-03-16T10:40:09.424 INFO:teuthology.orchestra.run.smithi194.stderr:real 11m0.339s 2024-03-16T10:40:09.424 INFO:teuthology.orchestra.run.smithi194.stderr:user 6m31.252s 2024-03-16T10:40:09.424 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m15.352s 2024-03-16T10:41:44.533 INFO:teuthology.orchestra.run.smithi039.stderr: 90.6% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-16T10:41:50.208 INFO:teuthology.orchestra.run.smithi039.stderr: 90.3% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-16T10:41:55.970 INFO:teuthology.orchestra.run.smithi039.stderr: 90.6% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-16T10:43:14.144 INFO:teuthology.orchestra.run.smithi039.stderr: 90.5% -- replaced with /var/log/ceph/65a28618-e37b-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-16T10:43:14.155 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-03-16T10:43:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:real 14m5.055s 2024-03-16T10:43:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:user 18m37.055s 2024-03-16T10:43:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:sys 2m6.620s 2024-03-16T10:43:14.156 INFO:tasks.cephadm:Archiving logs... 2024-03-16T10:43:14.157 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi039/log 2024-03-16T10:43:14.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T10:44:34.037 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi105/log 2024-03-16T10:44:34.056 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T10:45:36.493 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi194/log 2024-03-16T10:45:36.494 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T10:46:45.624 INFO:tasks.cephadm:Removing cluster... 2024-03-16T10:46:45.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force 2024-03-16T10:46:46.327 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:46:48.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force 2024-03-16T10:46:48.802 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:46:49.476 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65a28618-e37b-11ee-95c9-87774f69a715 --force 2024-03-16T10:46:49.770 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 65a28618-e37b-11ee-95c9-87774f69a715 2024-03-16T10:46:50.527 INFO:tasks.cephadm:Removing cephadm ... 2024-03-16T10:46:50.527 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T10:46:50.587 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T10:46:50.606 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T10:46:50.621 INFO:tasks.cephadm:Teardown complete 2024-03-16T10:46:50.622 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T10:46:50.633 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T10:46:50.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T10:46:50.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T10:46:50.649 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T10:46:50.852 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T10:46:50.863 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T10:46:50.863 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:46:50.871 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:46:50.873 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:46:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-16T10:46:50.887 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-16T10:46:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 +1221us[+1221us] +/- 96ms 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -671us[ -654us] +/- 52ms 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +59us[ +59us] +/- 66ms 2024-03-16T10:46:50.993 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:46:50.995 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T10:46:50.995 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-16T10:46:50.996 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 +1282us[+1270us] +/- 96ms 2024-03-16T10:46:50.996 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -676us[ -688us] +/- 52ms 2024-03-16T10:46:50.996 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +65us[ +53us] +/- 66ms 2024-03-16T10:46:50.996 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:46:51.030 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T10:46:51.031 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-03-16T10:46:51.031 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +1191us[+1191us] +/- 96ms 2024-03-16T10:46:51.031 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -702us[ -710us] +/- 52ms 2024-03-16T10:46:51.031 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +34us[ +26us] +/- 66ms 2024-03-16T10:46:51.031 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:46:51.034 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T10:46:51.046 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T10:46:51.046 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T10:46:51.058 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T10:46:51.379 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T10:46:51.485 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T10:46:51.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T10:46:51.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:51.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T10:46:51.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:51.741 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T10:46:51.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:51.772 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T10:46:51.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T10:46:51.792 INFO:teuthology.task.internal:Duration was 3603.392105 seconds 2024-03-16T10:46:51.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T10:46:51.805 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T10:46:51.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T10:46:51.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T10:46:51.815 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T10:46:51.875 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T10:46:51.886 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T10:46:52.102 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T10:46:52.793 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T10:46:52.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-03-16T10:46:52.794 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T10:46:52.867 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-16T10:46:52.867 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T10:46:52.902 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-03-16T10:46:52.902 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T10:46:52.937 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T10:46:52.937 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T10:46:52.945 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T10:46:52.948 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T10:46:53.050 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T10:46:53.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T10:46:53.055 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T10:46:53.058 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T10:46:53.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T10:46:53.616 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T10:46:53.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T10:46:53.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T10:46:53.808 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T10:46:53.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T10:46:53.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:53.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:53.872 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:53.912 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-16T10:46:53.930 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-03-16T10:46:54.019 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-03-16T10:46:54.084 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:54.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:54.136 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:54.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:54.151 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:46:54.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:46:54.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T10:46:54.176 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T10:46:54.178 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi039 2024-03-16T10:46:54.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T10:46:54.297 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi105 2024-03-16T10:46:54.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T10:46:54.336 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352/remote/smithi194 2024-03-16T10:46:54.336 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T10:46:54.376 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T10:46:54.377 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T10:46:54.385 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T10:46:54.387 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T10:46:54.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T10:46:54.449 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T10:46:54.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T10:46:54.460 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T10:46:54.460 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T10:46:54.468 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T10:46:54.470 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T10:46:54.482 INFO:teuthology.orchestra.run.smithi105.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 10:46 /home/ubuntu/cephtest 2024-03-16T10:46:54.485 INFO:teuthology.orchestra.run.smithi194.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 10:46 /home/ubuntu/cephtest 2024-03-16T10:46:54.520 INFO:teuthology.orchestra.run.smithi039.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 10:46 /home/ubuntu/cephtest 2024-03-16T10:46:54.523 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T10:46:54.536 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T10:46:54.559 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T10:46:54.585 DEBUG:teuthology.nuke:shortname: smithi039 2024-03-16T10:46:54.585 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:46:54.611 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-16T10:46:54.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:46:54.615 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.062763', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGax3KfyepI1aeFbFR+eZp0QEsHi3koD1PNm2ErUMi7rvCFiNHez0CQndEf2A36EKowle6bvdTOmp9sZynNTLr8='} 2024-03-16T10:46:54.636 DEBUG:teuthology.nuke:shortname: smithi194 2024-03-16T10:46:54.636 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:46:54.639 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.063897', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNS5RuJNA59ZTTXEREeHQDai/Ymcd64jQ6o72DmdFPGMmssVzPols1GzfpZL5Uv+VgPIBD/QmuP+bQEzaHd5LP8='} 2024-03-16T10:46:54.670 INFO:teuthology.orchestra.console.smithi039:Power off 2024-03-16T10:46:54.670 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T10:46:54.694 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2024-03-16T10:46:54.694 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:46:54.710 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-16T10:46:54.711 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T10:46:54.719 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-16T10:46:54.733 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-16T10:46:54.733 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:46:54.755 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-16T10:46:58.720 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:46:58.756 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:46:58.831 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-16T10:46:58.865 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-16T10:47:02.268 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602352', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:36:00.061366', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3p0FOGBulTj9OeasAvuO55RuNz3ACwJ1Ht4MX9AxJ5OyYQhEEjkH3SJUwBMDc+ejCTO72NgDdspvgyYucxxl4='} 2024-03-16T10:47:02.319 INFO:teuthology.orchestra.console.smithi194:Power off 2024-03-16T10:47:02.319 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T10:47:02.344 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-03-16T10:47:02.344 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:02.368 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-16T10:47:02.832 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:02.866 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:02.943 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-16T10:47:02.974 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-16T10:47:03.075 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-16T10:47:06.370 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:06.480 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-16T10:47:06.944 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:07.055 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-03-16T10:47:07.156 INFO:teuthology.orchestra.console.smithi039:Power off completed 2024-03-16T10:47:10.483 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:47:10.593 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-03-16T10:47:10.693 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-03-16T10:47:10.798 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3845b7ac715a3ff198582638d199a90e30e254da description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} duration: 3603.3921048641205 failure_reason: '"2024-03-16T09:59:20.780305+0000 mon.a (mon.0) 210 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-16T10:47:10.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T10:47:10.892 INFO:teuthology.run:FAIL